-- Logs begin at Wed 2022-09-21 11:39:21 UTC, end at Wed 2022-09-21 12:46:54 UTC. -- Sep 21 12:05:41.327468 dv-cinder-858678-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 21 12:05:41.336216 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 21 12:05:41 2022] *** Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: nodename: dv-cinder-858678-1-iscsi Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: machine: x86_64 Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: clock source: unix Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: pcre jit disabled Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: detected number of CPU cores: 4 Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: current working directory: / Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: detected binary path: /usr/bin/uwsgi-core Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: your processes number limit is 47680 Sep 21 12:05:41.338228 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: your memory page size is 4096 bytes Sep 21 12:05:41.339669 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: detected max file descriptor number: 2048 Sep 21 12:05:41.339669 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: lock engine: pthread robust mutexes Sep 21 12:05:41.339669 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: thunder lock: enabled Sep 21 12:05:41.339669 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 21 12:05:41.339669 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 21 12:05:41.378798 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: Python main interpreter initialized at 0x564188ca79c0 Sep 21 12:05:41.378798 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: python threads support enabled Sep 21 12:05:41.378798 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: your server socket listen backlog is limited to 100 connections Sep 21 12:05:41.378798 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: your mercy for graceful operations on workers is 90 seconds Sep 21 12:05:41.379153 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: mapped 403077 bytes (393 KB) for 2 cores Sep 21 12:05:41.379235 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: *** Operational MODE: preforking *** Sep 21 12:05:41.379495 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: *** uWSGI is running in multiple interpreter mode *** Sep 21 12:05:41.379495 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: spawned uWSGI master process (pid: 81395) Sep 21 12:05:41.379697 dv-cinder-858678-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 21 12:05:41.381083 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: spawned uWSGI worker 1 (pid: 81406, cores: 1) Sep 21 12:05:41.381083 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: spawned uWSGI worker 2 (pid: 81407, cores: 1) Sep 21 12:05:41.381820 dv-cinder-858678-1-iscsi devstack@keystone.service[81395]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 21 12:05:45.661816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 12:05:45.661816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: from cryptography.utils import int_from_bytes Sep 21 12:05:45.662683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 12:05:45.662683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: from cryptography.utils import int_from_bytes Sep 21 12:05:45.662833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 12:05:45.662833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: from cryptography.utils import int_from_bytes Sep 21 12:05:45.662833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 12:05:45.662833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: from cryptography.utils import int_from_bytes Sep 21 12:05:45.711657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 21 12:05:45.711876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 21 12:05:45.712064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] command line args: [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 21 12:05:45.712260 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 21 12:05:45.712352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 21 12:05:45.712456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 21 12:05:45.712567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ================================================================================ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 21 12:05:45.712649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] command line args: [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 21 12:05:45.712723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] admin_token = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.712909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 21 12:05:45.713007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_dir = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.713090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ================================================================================ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 21 12:05:45.713172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.713313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] config_source = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.713398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] admin_token = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.713476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] debug = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.713547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config_dir = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.713672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.714187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.714187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] config_source = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.714894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] debug = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.714975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.715046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.715111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.715294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.715381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.715511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.715598 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.715717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.715829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.715950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_config_append = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716241 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_dir = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_config_append = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_dir = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.716954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717441 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.717924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.718051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.718133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.718251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.718365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.718443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.718591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.718684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.718822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.718899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.719057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.719181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.719291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.719379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.719527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.719645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.719794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.719925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.720042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.720290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.720411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.720532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] publish_errors = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.720687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.720812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.720919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.721031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] publish_errors = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.721149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.721271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.721392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.721512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.721625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.721759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.721883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] standard_threads = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.722003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.722117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.722236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.722356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.722479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] standard_threads = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.722600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.722714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.722828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_journal = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.722946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.723061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_json = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.723240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.723364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_stderr = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.723481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_journal = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.723601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] use_syslog = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.723717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_json = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.723846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.723975 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_stderr = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.724091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.724204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] use_syslog = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.724321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.724434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 12:05:45.724548 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.724669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.724785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.724899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.725012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.725130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.725245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.725367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.725489 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.725723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.725837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.725951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.726073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.726188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.726302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.726415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.726527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.726642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.726762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.726877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.726981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.727099 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.727220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.727334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.727456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.727575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.727691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.727869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.727989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.728102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.728225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.728347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.728461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.728575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.728687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.728880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.728997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.729110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.729223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.729336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.729449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.729574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.729710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.729826 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.729939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.730052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.730165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.730289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.730405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.730518 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.730621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.730721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.730817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.730942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.731057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.731170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.731283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.731401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.731516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.731638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.731765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.731883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.731996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.732109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.732222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.732346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.732470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.732585 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.732698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.732817 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.732933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.733056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.733177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.733297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.733417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.733532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.733645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.733766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.733882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.733996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.734109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.734222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.734339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.734528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.734651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.734772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.734887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.735001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.735115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.735240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.735356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.735469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.735588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.735703 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.735866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.735994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.736117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.736232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.736346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.736459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.736572 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.736678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.736775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.736876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.736976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.737072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.737175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.737272 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.737368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.737469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.737602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.737728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.737853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.737969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.738089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.738202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.738316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.738430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.738552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.738668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.738788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.738901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.739022 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.739137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.739263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.739391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.739521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.739638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.739764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.739882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.739997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.740102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.740216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.740320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.740423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.740524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.740626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.740728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.740839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.740936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741826 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.741922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.742927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.743029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.743126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.743232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.743374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.743374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.743576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.743576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.743819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.743982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.744106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.744221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.744340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.744448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.744552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.744674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.744790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.744906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.745021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.745021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.745236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.745429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.745548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.745663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.745784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.745900 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.746028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.746028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.746264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.746387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.746503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.746620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.746734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.746849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.746971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.747088 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.747203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.747324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.747440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.747555 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.747686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.747816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.747934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.748060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.748178 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.748293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.748417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.748534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.748650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.748774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.748891 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.external = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.749033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.749159 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.749283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.749412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.749531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.external = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.749647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.749764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.749881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.password = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.749881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] auth.token = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.750118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.750118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.750339 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.750455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.password = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.750577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.750694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] auth.token = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.750819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.750937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.751053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.751169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.751285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.751408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.751533 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.751651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.751832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.751952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.752068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.752184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.752308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.752433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.752551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.752675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.752793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.752910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.753035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.753152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.753269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.753385 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.753547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.753668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.753778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.753896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.754013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.754129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.754245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.754364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.754489 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.754607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.754723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.754840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.754965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.755083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.755208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.755327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.755443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.755559 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.755675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.755804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.755934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.755934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.756155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.756271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.756387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.756511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.756629 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.756745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.756861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.756976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.757093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.757641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.757754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.757754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.757937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.757937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.758989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.759086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.759196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.759294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.759471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.759573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.759670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.759850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.759956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.760055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.760167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.760290 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.760408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.760525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.760641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.760757 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.760883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.761548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.761708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.761827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.761945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.762084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.762213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.762332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.762449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.762566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.762691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.762809 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.762935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.763053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.763169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.763286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.763404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.763520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.763645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.764317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.764446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.764564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.764681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.764812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.764933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.765049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.765165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.765281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.765398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.765529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.765648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.765765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.765882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.765999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.766115 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.766243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.766362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.766481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.766597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.766713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.766839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.766957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.767074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.767191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.767307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.767423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.767549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.767668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.767799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.767919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.768036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.768153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.768282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.768402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.768519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.768636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.768753 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.768870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.768996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.769114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.769231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.769348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.769464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.769581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.769708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.769827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.769943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.770060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.770060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.770281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.770406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.770525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.770642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.770758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.770875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.770992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.771117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.771244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.771363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.771479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.771479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.771700 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.771872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.771992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.772110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.772225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.772341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.772458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.772582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.772701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.772819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.772935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.773051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.773167 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.773292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.773410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.773527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.773642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.773759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.773876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.774001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.774119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.774236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.774356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.774473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.774572 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.774697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.774815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.774932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.775048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.775164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.775280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.775404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.775522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.775638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.775768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.775888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.776005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.776138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.776256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.776373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.776489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.776628 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.776777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.777422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.777542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.777658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.777773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.777895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.778014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.778139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.778256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.778371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.778486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.778601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.778717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.778844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.778980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.779100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.779216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.779331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.779430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.779537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.779637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.779734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.779857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.779957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.780055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.780166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.780267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.780385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.780504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.780621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.780737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.780870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.780989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.781106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.781222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.781339 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.781462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.781589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.781707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.781823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.781940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.782056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.782173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.782297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.782416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.782533 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.782649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.782765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.782890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.783008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.783132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.783251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.783251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.783470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.783669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.783803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.783922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.784039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.784156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.784274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.784387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.784515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.784515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.784743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.784861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.784977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.785102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.785220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.785337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.785453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.785570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.785687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.785811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.785934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.785934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.786151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.786267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.786383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.786507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.786625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.786741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.786857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.786972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.787088 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.787211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.787328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.787328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.787547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.787646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.787742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.787923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788395 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788720 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.788954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789185 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789826 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.789968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790120 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790441 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.790915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.791030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.791145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.791268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.791386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.791502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.791618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.791733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.791875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.792006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.792124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.792285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.792490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.792673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.792797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.792923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.793041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.793157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.793272 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.793387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.793503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.793627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.793747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.793863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.793978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.794093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.794208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.794332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.794454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.794571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.794687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.794804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.794919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.795031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.795161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.795278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.795394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.795510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.795625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.795740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.795892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.796011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.796136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.796261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.796379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.796495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.796610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.796725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.796849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.796967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.797083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.797198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.797314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.797430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.797554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.797672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.797788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.797904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.798019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.798141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.798266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.798385 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.798505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.798621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.798737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.798852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.798978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.799095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.799211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.799327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.799450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.799566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.799693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.799907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.800027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.800027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 21 12:05:45.800245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.800245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.800245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.800245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.800245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 12:05:45.800245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 21 12:05:45.828101 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.828320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.828526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.829388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.829623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.829846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.836995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.838306 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.865276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.865506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.866529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.866764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.920196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.920506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.969334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.969572 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.970183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:45.970419 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.126270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.126506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.126723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.126944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.127053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.127162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.127246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.127377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.127459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.127646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.127890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.128106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.150437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.150682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.150844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.150945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.151079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.151162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.151303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.151385 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.151512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.152112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.152269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.152353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.152485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.152699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.163421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.163855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.176714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.176893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.177123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.177209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.177347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.177418 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.177563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.177645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.177775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.177857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.177987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.178068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.178199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.178415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 12:05:46.194674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.194886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 12:05:46.195225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.195346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.195429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 12:05:46.195554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 12:05:46.195679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.195921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.196018 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.196114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 12:05:46.196208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.196393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.196490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.196653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.196906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.197162 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.206506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.206780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.207049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.207182 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.207331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.207465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.207735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.208042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.215055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.215886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.217972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.218258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.218541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.218744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.218833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.219026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.219117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.219309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.219389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.219593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.219673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.219890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.219973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.220150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.220229 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.220418 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.220510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.220686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.220767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.220954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.221224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.221490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.242953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.243726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.244450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.244739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.245208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.245517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.249396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.249683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.249968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.250156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.250440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.250723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.256262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.256553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.256996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.257285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.263595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.263916 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.264000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.264208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.264439 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.264596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.264713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.264830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.265874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.266524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.266635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.266718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.266794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.266868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.266941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.266941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.281600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.281894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.283344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.283632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.285743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.287434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.288978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.289278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.290640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.290935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.293546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.295159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.296707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.298232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.299949 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.300252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.300539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.300833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.301117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.301401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.301492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.301713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.301999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.302287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.302573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.302856 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.316992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.317287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.317556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.317822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.318278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.318575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.318846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.319112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.325123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.326487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.326661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.328045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.328181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.328456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.329575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.329853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.331179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.331461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.331748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.332054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.332440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.332541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.332858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.333116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.333401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.333791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.344304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.344589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.344868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.345144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.345522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.345610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.345807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.345892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.346085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.346167 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.346418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.346803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.347083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.347390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.361200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.362577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.364098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.365571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.367061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.368544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.368661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.369942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.370027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.370297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.370720 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.371432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.371717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.372148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.377033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.378450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.378581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.379912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.381342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.381733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.382020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.382296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.382750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.383148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.383435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.383716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.390735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.391025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.391306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.394184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.394184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.394184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.398166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.398453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.398771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.399005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.399275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.399449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 12:05:46.399558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.399737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.399844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.400038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.400118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.400319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.400412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.400599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.400689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.400884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.400974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.401163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.401244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.401438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.401526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.402253 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.402350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.402538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.402812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.403098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.403367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.403654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 12:05:46.436147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564188ca79c0 pid: 81406 (default app) Sep 21 12:05:46.437446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564188ca79c0 pid: 81407 (default app) Sep 21 12:05:46.439475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22c702-faba-463f-8017-30ce5b477914 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:46.439789 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22c702-faba-463f-8017-30ce5b477914 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:46.440054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22c702-faba-463f-8017-30ce5b477914 None None] PATH_INFO: `/v3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:46.444192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1/1] 10.222.0.71 () {50 vars in 791 bytes} [Wed Sep 21 12:05:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:05:53.275643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbe51e-e1c6-454e-9f1b-d69b10cd50bc None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:53.276210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbe51e-e1c6-454e-9f1b-d69b10cd50bc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:53.276300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbe51e-e1c6-454e-9f1b-d69b10cd50bc None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:53.281067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1/2] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:05:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:05:53.286737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:53.287261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:53.287354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:53.358524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: INFO dbcounter [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] Registered counter for database keystone Sep 21 12:05:53.378801 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG oslo_db.sqlalchemy.engines [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81406) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 21 12:05:53.385138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writer thread running {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 21 12:05:53.564397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG passlib.handlers.bcrypt [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81406) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 21 12:05:53.564797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG passlib.handlers.bcrypt [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81406) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 21 12:05:53.624683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:05:53.626324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:05:53.759951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 2/3] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:05:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 21 12:05:54.240049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbe51e-e1c6-454e-9f1b-d69b10cd50bc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:05:54.309760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: INFO dbcounter [None req-eedbe51e-e1c6-454e-9f1b-d69b10cd50bc None None] Registered counter for database keystone Sep 21 12:05:54.333268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG oslo_db.sqlalchemy.engines [None req-eedbe51e-e1c6-454e-9f1b-d69b10cd50bc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81407) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 21 12:05:54.339879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writer thread running {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 21 12:05:54.458015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbe51e-e1c6-454e-9f1b-d69b10cd50bc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:05:54.462400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ae089-c866-46a7-bca2-b512b7db87f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:05:54.463031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ae089-c866-46a7-bca2-b512b7db87f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:54.463313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ae089-c866-46a7-bca2-b512b7db87f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:54.463584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ae089-c866-46a7-bca2-b512b7db87f9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:54.472171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ae089-c866-46a7-bca2-b512b7db87f9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:05:54.524447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WARNING keystone.common.rbac_enforcer.enforcer [None req-735ae089-c866-46a7-bca2-b512b7db87f9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 21 12:05:54.524740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ae089-c866-46a7-bca2-b512b7db87f9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:05:54.534441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 2/4] 10.222.0.71 () {58 vars in 1324 bytes} [Wed Sep 21 12:05:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 12:05:54.540461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:05:54.557686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3adcdb-30e4-4d21-90c7-65b10f354ab0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:05:54.562770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c620ea-0909-4799-a113-c06d4f2b271d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:05:54.563901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c620ea-0909-4799-a113-c06d4f2b271d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:54.564188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c620ea-0909-4799-a113-c06d4f2b271d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:54.564462 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c620ea-0909-4799-a113-c06d4f2b271d admin admin] PATH_INFO: `/v3/projects/5d4deb4c3b0e444f92c029c040b5603c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:54.566059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c620ea-0909-4799-a113-c06d4f2b271d admin admin] RBAC: Authorizing `identity:get_project(project_id=5d4deb4c3b0e444f92c029c040b5603c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:05:54.619582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WARNING keystone.common.rbac_enforcer.enforcer [None req-93c620ea-0909-4799-a113-c06d4f2b271d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 21 12:05:54.619957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c620ea-0909-4799-a113-c06d4f2b271d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:05:54.625379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 3/5] 10.222.0.71 () {56 vars in 1207 bytes} [Wed Sep 21 12:05:54 2022] GET /identity/v3/projects/5d4deb4c3b0e444f92c029c040b5603c => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:05:56.786431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520c9b5-0011-4820-99c3-7a1757b3ed47 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:56.786880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520c9b5-0011-4820-99c3-7a1757b3ed47 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:56.786981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520c9b5-0011-4820-99c3-7a1757b3ed47 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:56.791049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 3/6] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:05:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:05:56.796870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12074139-2edf-4d2b-a355-02b4f451e503 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:56.797259 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12074139-2edf-4d2b-a355-02b4f451e503 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:56.797417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12074139-2edf-4d2b-a355-02b4f451e503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:56.817666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-12074139-2edf-4d2b-a355-02b4f451e503 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:05:56.819193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12074139-2edf-4d2b-a355-02b4f451e503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:05:56.826978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 4/7] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:05:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 21 12:05:57.092153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520c9b5-0011-4820-99c3-7a1757b3ed47 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:05:57.104312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520c9b5-0011-4820-99c3-7a1757b3ed47 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:05:57.109052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda28f00-f443-403b-b95c-a10eb5dc53d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:05:57.109651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda28f00-f443-403b-b95c-a10eb5dc53d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:57.109934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda28f00-f443-403b-b95c-a10eb5dc53d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:57.110209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda28f00-f443-403b-b95c-a10eb5dc53d7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:57.117810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda28f00-f443-403b-b95c-a10eb5dc53d7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:05:57.119574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda28f00-f443-403b-b95c-a10eb5dc53d7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:05:57.128304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 4/8] 10.222.0.71 () {58 vars in 1324 bytes} [Wed Sep 21 12:05:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:05:57.133766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12074139-2edf-4d2b-a355-02b4f451e503 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:05:57.142799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12074139-2edf-4d2b-a355-02b4f451e503 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:05:57.147399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d461d6-5275-4d8e-9c3e-b28e524dfbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:05:57.148045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d461d6-5275-4d8e-9c3e-b28e524dfbb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:57.148324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d461d6-5275-4d8e-9c3e-b28e524dfbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:57.148592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d461d6-5275-4d8e-9c3e-b28e524dfbb2 admin admin] PATH_INFO: `/v3/users/661f5d19be0f436ca14e6e9b440ca709` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:57.149889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d461d6-5275-4d8e-9c3e-b28e524dfbb2 admin admin] RBAC: Authorizing `identity:get_user(user_id=661f5d19be0f436ca14e6e9b440ca709)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:05:57.152266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d461d6-5275-4d8e-9c3e-b28e524dfbb2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:05:57.153797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 5/9] 10.222.0.71 () {56 vars in 1198 bytes} [Wed Sep 21 12:05:57 2022] GET /identity/v3/users/661f5d19be0f436ca14e6e9b440ca709 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:05:59.348173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba88233-df7b-426b-80ff-beedc3762ec7 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:59.348727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba88233-df7b-426b-80ff-beedc3762ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:59.348727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba88233-df7b-426b-80ff-beedc3762ec7 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:59.350067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 5/10] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:05:59.355542 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee208e9-3c46-413c-9128-9a9550577493 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:59.355959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee208e9-3c46-413c-9128-9a9550577493 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:59.356123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee208e9-3c46-413c-9128-9a9550577493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:59.374281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5ee208e9-3c46-413c-9128-9a9550577493 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:05:59.375443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5ee208e9-3c46-413c-9128-9a9550577493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:05:59.421300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e60e80-c25a-4d41-b96b-0807f02f2ed1 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:59.421562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e60e80-c25a-4d41-b96b-0807f02f2ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:59.421924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e60e80-c25a-4d41-b96b-0807f02f2ed1 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:59.424169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 6/11] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:05:59.429938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337e63c-1fb8-4e43-b71d-89e366ac109a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:59.431321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337e63c-1fb8-4e43-b71d-89e366ac109a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:59.431321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337e63c-1fb8-4e43-b71d-89e366ac109a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:59.459676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 6/12] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:05:59.474476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG passlib.handlers.bcrypt [None req-c337e63c-1fb8-4e43-b71d-89e366ac109a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81407) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 21 12:05:59.474875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG passlib.handlers.bcrypt [None req-c337e63c-1fb8-4e43-b71d-89e366ac109a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81407) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 21 12:05:59.524372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c337e63c-1fb8-4e43-b71d-89e366ac109a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:05:59.525995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c337e63c-1fb8-4e43-b71d-89e366ac109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:05:59.637448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 7/13] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:05:59.743606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee208e9-3c46-413c-9128-9a9550577493 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:05:59.768699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:05:59.849795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee208e9-3c46-413c-9128-9a9550577493 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:05:59.883323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bda936-bcda-4d48-b6c7-452613c0a833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:05:59.883990 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bda936-bcda-4d48-b6c7-452613c0a833 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:05:59.884268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bda936-bcda-4d48-b6c7-452613c0a833 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:05:59.884540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bda936-bcda-4d48-b6c7-452613c0a833 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:05:59.904892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bda936-bcda-4d48-b6c7-452613c0a833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:05:59.906487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337e63c-1fb8-4e43-b71d-89e366ac109a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-68bda936-bcda-4d48-b6c7-452613c0a833 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:05:59.918767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:05:59.920803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:05:59.921906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 7/14] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:05:59.926196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bda936-bcda-4d48-b6c7-452613c0a833 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:00.009505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:00.022158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337e63c-1fb8-4e43-b71d-89e366ac109a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:00.024435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:00.028862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bda936-bcda-4d48-b6c7-452613c0a833 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:00.062690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b036f7-4eb6-482f-b6c1-b8e6b9e36af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:00.063163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b548e-b191-4171-99e6-cfa7daae284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:00.063578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b036f7-4eb6-482f-b6c1-b8e6b9e36af6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:00.063837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b548e-b191-4171-99e6-cfa7daae284e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:00.063943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b036f7-4eb6-482f-b6c1-b8e6b9e36af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:00.064129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b548e-b191-4171-99e6-cfa7daae284e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:00.064216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b036f7-4eb6-482f-b6c1-b8e6b9e36af6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:00.064407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b548e-b191-4171-99e6-cfa7daae284e None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:00.065078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b548e-b191-4171-99e6-cfa7daae284e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:00.065383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b548e-b191-4171-99e6-cfa7daae284e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:00.066780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b548e-b191-4171-99e6-cfa7daae284e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:00.078431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 8/15] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:05:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:00.084099 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b548e-b191-4171-99e6-cfa7daae284e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:00.131319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b036f7-4eb6-482f-b6c1-b8e6b9e36af6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:00.132877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b036f7-4eb6-482f-b6c1-b8e6b9e36af6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:00.182759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b548e-b191-4171-99e6-cfa7daae284e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:00.216712 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38b313-e38d-4244-b636-5487ec1b00f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:00.217348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38b313-e38d-4244-b636-5487ec1b00f2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:00.217629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38b313-e38d-4244-b636-5487ec1b00f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:00.217911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38b313-e38d-4244-b636-5487ec1b00f2 None admin] PATH_INFO: `/v3/users/661f5d19be0f436ca14e6e9b440ca709` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:00.219064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38b313-e38d-4244-b636-5487ec1b00f2 None admin] RBAC: Authorizing `identity:get_user(user_id=661f5d19be0f436ca14e6e9b440ca709)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:00.220637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b38b313-e38d-4244-b636-5487ec1b00f2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:00.222510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 9/16] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:00 2022] GET /identity/v3/users/661f5d19be0f436ca14e6e9b440ca709 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:00.228509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38b313-e38d-4244-b636-5487ec1b00f2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:00.231015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 8/17] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 12:06:00.236881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b036f7-4eb6-482f-b6c1-b8e6b9e36af6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:00.240031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:00.248030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:00.345052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b036f7-4eb6-482f-b6c1-b8e6b9e36af6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:00.346567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b38b313-e38d-4244-b636-5487ec1b00f2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:00.378500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e51edd-d05c-4d17-b408-afc21012ada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:00.379347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e51edd-d05c-4d17-b408-afc21012ada8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:00.379625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e51edd-d05c-4d17-b408-afc21012ada8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:00.379929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e51edd-d05c-4d17-b408-afc21012ada8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:00.381766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1788e-c015-4b17-a0e5-c9d44a1155aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:00.382393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1788e-c015-4b17-a0e5-c9d44a1155aa None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:00.382678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1788e-c015-4b17-a0e5-c9d44a1155aa None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:00.382953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1788e-c015-4b17-a0e5-c9d44a1155aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:00.384108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1788e-c015-4b17-a0e5-c9d44a1155aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:00.385566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1788e-c015-4b17-a0e5-c9d44a1155aa None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:00.387179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 10/18] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:00.392919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1788e-c015-4b17-a0e5-c9d44a1155aa None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:00.398386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e51edd-d05c-4d17-b408-afc21012ada8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:00.399998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e51edd-d05c-4d17-b408-afc21012ada8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-26e51edd-d05c-4d17-b408-afc21012ada8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:06:00.420017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:00.421027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:06:00.421811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:06:00.421811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:06:00.421811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:00.421811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 9/19] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:06:00.426785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e51edd-d05c-4d17-b408-afc21012ada8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:00.475299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:00.484638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:00.496694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1788e-c015-4b17-a0e5-c9d44a1155aa None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:00.534022 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e51edd-d05c-4d17-b408-afc21012ada8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:00.534220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc3778-f538-4181-a6ee-596d5f21e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:00.534634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3778-f538-4181-a6ee-596d5f21e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:00.534919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3778-f538-4181-a6ee-596d5f21e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:00.535201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc3778-f538-4181-a6ee-596d5f21e1c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:00.535970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3778-f538-4181-a6ee-596d5f21e1c3 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.domain.id=default, user.id=661f5d19be0f436ca14e6e9b440ca709) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:00.536283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3778-f538-4181-a6ee-596d5f21e1c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:00.537735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc3778-f538-4181-a6ee-596d5f21e1c3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:00.545485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 11/20] 10.222.0.71 () {56 vars in 1322 bytes} [Wed Sep 21 12:06:00 2022] GET /identity/v3/role_assignments?user.id=661f5d19be0f436ca14e6e9b440ca709&scope.domain.id=default&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:00.577479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ed926-ece7-467f-b352-93edef2ceae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:00.578114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ed926-ece7-467f-b352-93edef2ceae3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:00.578400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ed926-ece7-467f-b352-93edef2ceae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:00.578676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ed926-ece7-467f-b352-93edef2ceae3 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:00.579340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ed926-ece7-467f-b352-93edef2ceae3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:00.579647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ed926-ece7-467f-b352-93edef2ceae3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:00.581426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ed926-ece7-467f-b352-93edef2ceae3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:00.601784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 10/21] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:03.155511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f7373-32bb-43e3-a0ed-31e20356e861 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:03.156047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f7373-32bb-43e3-a0ed-31e20356e861 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:03.156047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f7373-32bb-43e3-a0ed-31e20356e861 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:03.157401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 12/22] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:03.175952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b282-cfaa-4de2-ae22-dc8e3ab6bda2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:03.176465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b282-cfaa-4de2-ae22-dc8e3ab6bda2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:03.176538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b282-cfaa-4de2-ae22-dc8e3ab6bda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:03.180137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5f758-7cac-429a-a09d-902d9d99a428 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:03.180678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5f758-7cac-429a-a09d-902d9d99a428 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:03.181154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5f758-7cac-429a-a09d-902d9d99a428 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:03.183673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 13/23] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:03.188403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15919d-2115-4569-8898-5ca1468f0641 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:03.188960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15919d-2115-4569-8898-5ca1468f0641 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:03.189452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15919d-2115-4569-8898-5ca1468f0641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:03.221163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ff15919d-2115-4569-8898-5ca1468f0641 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:03.222371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ff15919d-2115-4569-8898-5ca1468f0641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:03.228375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7835b282-cfaa-4de2-ae22-dc8e3ab6bda2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:03.229591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7835b282-cfaa-4de2-ae22-dc8e3ab6bda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:03.411973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 14/24] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:03.415890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 11/25] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:03.439307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150a8a6-ce2a-4a79-a47e-9fe298cdbd2b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:03.439857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150a8a6-ce2a-4a79-a47e-9fe298cdbd2b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:03.440271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150a8a6-ce2a-4a79-a47e-9fe298cdbd2b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:03.442079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 15/26] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:03.447290 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76221a-1111-41df-81ad-f01f4466e936 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:03.447801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76221a-1111-41df-81ad-f01f4466e936 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:03.448230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76221a-1111-41df-81ad-f01f4466e936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:03.476774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0f76221a-1111-41df-81ad-f01f4466e936 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:03.478193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0f76221a-1111-41df-81ad-f01f4466e936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:03.527591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:03.560469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 12/27] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:03.779453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150a8a6-ce2a-4a79-a47e-9fe298cdbd2b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:03.863067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76221a-1111-41df-81ad-f01f4466e936 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:03.924140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:03.976637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150a8a6-ce2a-4a79-a47e-9fe298cdbd2b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:03.998165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76221a-1111-41df-81ad-f01f4466e936 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:04.016165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca92ca4-a82c-4ca2-b3cb-20558d639bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:04.016701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca92ca4-a82c-4ca2-b3cb-20558d639bbe None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.016984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca92ca4-a82c-4ca2-b3cb-20558d639bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.017261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca92ca4-a82c-4ca2-b3cb-20558d639bbe None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.017943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca92ca4-a82c-4ca2-b3cb-20558d639bbe None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:04.019583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca92ca4-a82c-4ca2-b3cb-20558d639bbe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:04.040197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b09adb-6634-4200-98d2-96dcf4b10a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:04.041158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b09adb-6634-4200-98d2-96dcf4b10a39 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.041620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b09adb-6634-4200-98d2-96dcf4b10a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.042082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b09adb-6634-4200-98d2-96dcf4b10a39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.056884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b09adb-6634-4200-98d2-96dcf4b10a39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c5b09adb-6634-4200-98d2-96dcf4b10a39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:04.063922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:04.064774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:04.065517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:04.067871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 13/28] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:06:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:06:04.069237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff092943-236f-453b-a627-056d7445a084 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.069697 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff092943-236f-453b-a627-056d7445a084 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.070214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff092943-236f-453b-a627-056d7445a084 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.071619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 14/29] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:04.072514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff092943-236f-453b-a627-056d7445a084 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:04.112128 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:04.158716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 16/30] 10.222.0.71 () {60 vars in 1131 bytes} [Wed Sep 21 12:06:03 2022] POST /identity/v3/roles => generated 219 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:04.159723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca92ca4-a82c-4ca2-b3cb-20558d639bbe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:04.184213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff092943-236f-453b-a627-056d7445a084 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:04.225775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1d1ac-7c77-4f99-8255-c9386c8a6c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:04.226929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1d1ac-7c77-4f99-8255-c9386c8a6c6a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.227797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1d1ac-7c77-4f99-8255-c9386c8a6c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.228409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1d1ac-7c77-4f99-8255-c9386c8a6c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.241462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1d1ac-7c77-4f99-8255-c9386c8a6c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:04.244121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1d1ac-7c77-4f99-8255-c9386c8a6c6a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-7cf1d1ac-7c77-4f99-8255-c9386c8a6c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:06:04.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:04.269486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:06:04.270281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:06:04.270281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:06:04.270281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:04.272504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 15/31] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:04.274678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894d15f-2149-412a-916c-19c95bc34ef2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.275191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894d15f-2149-412a-916c-19c95bc34ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.275672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894d15f-2149-412a-916c-19c95bc34ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.280211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca92ca4-a82c-4ca2-b3cb-20558d639bbe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:04.298436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2894d15f-2149-412a-916c-19c95bc34ef2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:04.300634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2894d15f-2149-412a-916c-19c95bc34ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:04.319902 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5f9e-30ba-451b-9ed7-4da7b80ff151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:04.320547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5f9e-30ba-451b-9ed7-4da7b80ff151 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.320879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5f9e-30ba-451b-9ed7-4da7b80ff151 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.321155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5f9e-30ba-451b-9ed7-4da7b80ff151 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.321903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5f9e-30ba-451b-9ed7-4da7b80ff151 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:04.322264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5f9e-30ba-451b-9ed7-4da7b80ff151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:04.324666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5f9e-30ba-451b-9ed7-4da7b80ff151 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:04.326471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:04.335312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 17/32] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:04.336281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5f9e-30ba-451b-9ed7-4da7b80ff151 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:04.398056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 16/33] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:04.399317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894d15f-2149-412a-916c-19c95bc34ef2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:04.439882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:04.493776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5f9e-30ba-451b-9ed7-4da7b80ff151 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:04.562029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150b7f1-ac8b-4428-b7c3-2270d1859449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:04.562897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150b7f1-ac8b-4428-b7c3-2270d1859449 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.564128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150b7f1-ac8b-4428-b7c3-2270d1859449 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.565088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150b7f1-ac8b-4428-b7c3-2270d1859449 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.565886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150b7f1-ac8b-4428-b7c3-2270d1859449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:04.566724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150b7f1-ac8b-4428-b7c3-2270d1859449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:04.569288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150b7f1-ac8b-4428-b7c3-2270d1859449 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:04.603022 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 18/34] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:04.605459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894d15f-2149-412a-916c-19c95bc34ef2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:04.609590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150b7f1-ac8b-4428-b7c3-2270d1859449 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:04.680246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e23a7f-c75c-416c-b44c-06ade2003c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:04.681223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23a7f-c75c-416c-b44c-06ade2003c99 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.681643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23a7f-c75c-416c-b44c-06ade2003c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.682055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e23a7f-c75c-416c-b44c-06ade2003c99 None admin] PATH_INFO: `/v3/users/661f5d19be0f436ca14e6e9b440ca709` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.685958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e23a7f-c75c-416c-b44c-06ade2003c99 None admin] RBAC: Authorizing `identity:get_user(user_id=661f5d19be0f436ca14e6e9b440ca709)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:04.687731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e23a7f-c75c-416c-b44c-06ade2003c99 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:04.690890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 17/35] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:04 2022] GET /identity/v3/users/661f5d19be0f436ca14e6e9b440ca709 => generated 247 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:04.696426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e23a7f-c75c-416c-b44c-06ade2003c99 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:04.729948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:04.801300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150b7f1-ac8b-4428-b7c3-2270d1859449 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:04.836042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:04.842478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89dc92-75f7-4478-a796-3d7d2da764e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:04.843275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89dc92-75f7-4478-a796-3d7d2da764e5 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.843562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89dc92-75f7-4478-a796-3d7d2da764e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.843938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89dc92-75f7-4478-a796-3d7d2da764e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.844721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89dc92-75f7-4478-a796-3d7d2da764e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:04.846335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89dc92-75f7-4478-a796-3d7d2da764e5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:04.853552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e23a7f-c75c-416c-b44c-06ade2003c99 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:04.877896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 19/36] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:06:04 2022] POST /identity/v3/projects => generated 301 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:04.879779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521f3ef-720f-44fe-bf6f-8fe2bb8780fe None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.880159 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521f3ef-720f-44fe-bf6f-8fe2bb8780fe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.880766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521f3ef-720f-44fe-bf6f-8fe2bb8780fe None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.887525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 20/37] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:04.887525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521f3ef-720f-44fe-bf6f-8fe2bb8780fe None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:04.907405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466892d-74c0-4c28-98e0-2f1aa2df7b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:04.908453 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466892d-74c0-4c28-98e0-2f1aa2df7b70 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.908995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466892d-74c0-4c28-98e0-2f1aa2df7b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.909448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466892d-74c0-4c28-98e0-2f1aa2df7b70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.910999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466892d-74c0-4c28-98e0-2f1aa2df7b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:04.912772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466892d-74c0-4c28-98e0-2f1aa2df7b70 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:04.915715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 18/38] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:04.917814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9aadfe-3866-4a2b-94b1-2bec1d322807 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:04.918291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9aadfe-3866-4a2b-94b1-2bec1d322807 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:04.919269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9aadfe-3866-4a2b-94b1-2bec1d322807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:04.942393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bc9aadfe-3866-4a2b-94b1-2bec1d322807 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:04.944239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bc9aadfe-3866-4a2b-94b1-2bec1d322807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:05.039669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521f3ef-720f-44fe-bf6f-8fe2bb8780fe None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:05.053691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 19/39] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:05.053691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9aadfe-3866-4a2b-94b1-2bec1d322807 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:05.062051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:05.100228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b27cc0-54d9-4e64-9b92-2b44a08ba535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:05.101314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b27cc0-54d9-4e64-9b92-2b44a08ba535 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:05.101729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b27cc0-54d9-4e64-9b92-2b44a08ba535 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:05.102009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b27cc0-54d9-4e64-9b92-2b44a08ba535 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:05.102689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b27cc0-54d9-4e64-9b92-2b44a08ba535 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:05.105767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b27cc0-54d9-4e64-9b92-2b44a08ba535 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:05.127937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 21/40] 10.222.0.71 () {60 vars in 1131 bytes} [Wed Sep 21 12:06:04 2022] POST /identity/v3/roles => generated 225 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:05.130003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdb7ab-90b5-436f-abaf-75fe707d9c38 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:05.130304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdb7ab-90b5-436f-abaf-75fe707d9c38 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:05.130637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bdb7ab-90b5-436f-abaf-75fe707d9c38 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:05.133009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 22/41] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:05.140720 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18924cbf-2197-4487-b867-a7c5a4b1456f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:05.141545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18924cbf-2197-4487-b867-a7c5a4b1456f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:05.141827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18924cbf-2197-4487-b867-a7c5a4b1456f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:05.169235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-18924cbf-2197-4487-b867-a7c5a4b1456f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:05.170918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-18924cbf-2197-4487-b867-a7c5a4b1456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:05.180123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:05.208446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9aadfe-3866-4a2b-94b1-2bec1d322807 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:05.255895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92818d-d3d1-454c-bfcb-8bf9cfcf5776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:05.256633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92818d-d3d1-454c-bfcb-8bf9cfcf5776 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:05.256909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92818d-d3d1-454c-bfcb-8bf9cfcf5776 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:05.257179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92818d-d3d1-454c-bfcb-8bf9cfcf5776 None admin] PATH_INFO: `/v3/domains/default/users/661f5d19be0f436ca14e6e9b440ca709/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:05.260192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92818d-d3d1-454c-bfcb-8bf9cfcf5776 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=661f5d19be0f436ca14e6e9b440ca709, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:05.262387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92818d-d3d1-454c-bfcb-8bf9cfcf5776 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:05.273459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 23/42] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:05.357434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18924cbf-2197-4487-b867-a7c5a4b1456f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:05.404790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 20/43] 10.222.0.71 () {58 vars in 1361 bytes} [Wed Sep 21 12:06:05 2022] PUT /identity/v3/domains/default/users/661f5d19be0f436ca14e6e9b440ca709/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:05.480595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:05.485537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18924cbf-2197-4487-b867-a7c5a4b1456f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:05.561369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92818d-d3d1-454c-bfcb-8bf9cfcf5776 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:05.635033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5b448-613f-4c7b-9c34-2a95049fcc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:05.635601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5b448-613f-4c7b-9c34-2a95049fcc2c None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:05.635919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5b448-613f-4c7b-9c34-2a95049fcc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:05.636275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5b448-613f-4c7b-9c34-2a95049fcc2c None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:05.637094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5b448-613f-4c7b-9c34-2a95049fcc2c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:05.638693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5b448-613f-4c7b-9c34-2a95049fcc2c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:05.656508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 24/44] 10.222.0.71 () {60 vars in 1131 bytes} [Wed Sep 21 12:06:05 2022] POST /identity/v3/roles => generated 223 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:05.693909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92818d-d3d1-454c-bfcb-8bf9cfcf5776 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:05.731526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c72085-e6bd-4f4a-b4e2-27b803ebc097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:05.732230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c72085-e6bd-4f4a-b4e2-27b803ebc097 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:05.732511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c72085-e6bd-4f4a-b4e2-27b803ebc097 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:05.732783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c72085-e6bd-4f4a-b4e2-27b803ebc097 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:05.734061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c72085-e6bd-4f4a-b4e2-27b803ebc097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:05.735731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c72085-e6bd-4f4a-b4e2-27b803ebc097 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:05.740329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 21/45] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:05.746510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5b448-613f-4c7b-9c34-2a95049fcc2c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:05.861995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5b448-613f-4c7b-9c34-2a95049fcc2c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:05.901742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0d6af-ac91-4406-8085-b8474980c142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:05.902845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d6af-ac91-4406-8085-b8474980c142 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:05.903126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d6af-ac91-4406-8085-b8474980c142 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:05.903399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0d6af-ac91-4406-8085-b8474980c142 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:05.904093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0d6af-ac91-4406-8085-b8474980c142 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:05.905563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0d6af-ac91-4406-8085-b8474980c142 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:05.932063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 25/46] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:06:05 2022] POST /identity/v3/projects => generated 312 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:07.594469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cf042-edb1-4576-8f9d-c561218690a9 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:07.595016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cf042-edb1-4576-8f9d-c561218690a9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:07.595016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cf042-edb1-4576-8f9d-c561218690a9 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:07.596421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 22/47] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:07.601863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de998fd-dda8-4e8b-8fcb-f6984aba10e2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:07.602353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de998fd-dda8-4e8b-8fcb-f6984aba10e2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:07.602436 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de998fd-dda8-4e8b-8fcb-f6984aba10e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:07.621114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4de998fd-dda8-4e8b-8fcb-f6984aba10e2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:07.622393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4de998fd-dda8-4e8b-8fcb-f6984aba10e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:07.625578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:07.692282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 26/48] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:07.960902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cf042-edb1-4576-8f9d-c561218690a9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:07.978329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:08.069205 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cf042-edb1-4576-8f9d-c561218690a9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:08.094222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497725a-036a-4b3b-98ed-63ed2cdbb16f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:08.094505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497725a-036a-4b3b-98ed-63ed2cdbb16f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:08.094781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497725a-036a-4b3b-98ed-63ed2cdbb16f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:08.096167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 27/49] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:08.101183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b85dd-2045-4511-b07e-b9a9782d0456 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:08.101465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b85dd-2045-4511-b07e-b9a9782d0456 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:08.101739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b85dd-2045-4511-b07e-b9a9782d0456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:08.105342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b2764-113d-466e-ad10-b6cadbc53405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:08.105978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b2764-113d-466e-ad10-b6cadbc53405 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:08.106264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b2764-113d-466e-ad10-b6cadbc53405 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:08.106560 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b2764-113d-466e-ad10-b6cadbc53405 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:08.117811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b2764-113d-466e-ad10-b6cadbc53405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:08.119299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-354b85dd-2045-4511-b07e-b9a9782d0456 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:08.120537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-354b85dd-2045-4511-b07e-b9a9782d0456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3b6b2764-113d-466e-ad10-b6cadbc53405 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:08.123479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:08.124273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 23/50] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:08.129679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b2764-113d-466e-ad10-b6cadbc53405 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:08.189020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 28/51] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:08.199057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:08.229525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b2764-113d-466e-ad10-b6cadbc53405 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:08.262055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8743d-6d36-4ba3-8488-4408c4d838db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:08.262757 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8743d-6d36-4ba3-8488-4408c4d838db None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:08.263041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8743d-6d36-4ba3-8488-4408c4d838db None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:08.263323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8743d-6d36-4ba3-8488-4408c4d838db None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:08.264027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8743d-6d36-4ba3-8488-4408c4d838db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:08.264332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8743d-6d36-4ba3-8488-4408c4d838db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:08.265725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8743d-6d36-4ba3-8488-4408c4d838db None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:08.278080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 24/52] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:08.283866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b85dd-2045-4511-b07e-b9a9782d0456 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:08.351128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:08.383984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b85dd-2045-4511-b07e-b9a9782d0456 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:08.417448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbcc61-3b5b-4ca9-9c89-81d860a89524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:08.418082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbcc61-3b5b-4ca9-9c89-81d860a89524 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:08.418364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbcc61-3b5b-4ca9-9c89-81d860a89524 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:08.418642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbcc61-3b5b-4ca9-9c89-81d860a89524 None admin] PATH_INFO: `/v3/users/661f5d19be0f436ca14e6e9b440ca709` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:08.419778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbcc61-3b5b-4ca9-9c89-81d860a89524 None admin] RBAC: Authorizing `identity:get_user(user_id=661f5d19be0f436ca14e6e9b440ca709)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:08.421256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbcc61-3b5b-4ca9-9c89-81d860a89524 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:08.422841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 29/53] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:08 2022] GET /identity/v3/users/661f5d19be0f436ca14e6e9b440ca709 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:08.428402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8743d-6d36-4ba3-8488-4408c4d838db None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:08.461715 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbcc61-3b5b-4ca9-9c89-81d860a89524 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:08.528746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8743d-6d36-4ba3-8488-4408c4d838db None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:08.562130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbcc61-3b5b-4ca9-9c89-81d860a89524 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:08.563117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53cb45-0c55-43d6-a86b-fa2d4376c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:08.563795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53cb45-0c55-43d6-a86b-fa2d4376c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:08.564078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53cb45-0c55-43d6-a86b-fa2d4376c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:08.564348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53cb45-0c55-43d6-a86b-fa2d4376c1a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:08.565455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53cb45-0c55-43d6-a86b-fa2d4376c1a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:08.566902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53cb45-0c55-43d6-a86b-fa2d4376c1a6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:08.568510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 25/54] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:08.574808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53cb45-0c55-43d6-a86b-fa2d4376c1a6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:08.578712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:08.597983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049c223-c975-4452-bd3a-a9aa56419d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:08.598616 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049c223-c975-4452-bd3a-a9aa56419d0b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:08.598898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049c223-c975-4452-bd3a-a9aa56419d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:08.599170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049c223-c975-4452-bd3a-a9aa56419d0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:08.600314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049c223-c975-4452-bd3a-a9aa56419d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:08.601768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049c223-c975-4452-bd3a-a9aa56419d0b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:08.603334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 30/55] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:08.608832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049c223-c975-4452-bd3a-a9aa56419d0b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:08.617289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:08.717514 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049c223-c975-4452-bd3a-a9aa56419d0b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:08.753532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be56426-0d35-45d8-9f7a-5993a98055ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:08.754148 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be56426-0d35-45d8-9f7a-5993a98055ac None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:08.754435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be56426-0d35-45d8-9f7a-5993a98055ac None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:08.754709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be56426-0d35-45d8-9f7a-5993a98055ac None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:08.755376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be56426-0d35-45d8-9f7a-5993a98055ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:08.756854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be56426-0d35-45d8-9f7a-5993a98055ac None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:08.782498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53cb45-0c55-43d6-a86b-fa2d4376c1a6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:08.782980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 31/56] 10.222.0.71 () {60 vars in 1140 bytes} [Wed Sep 21 12:06:08 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:06:08.819395 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc46f8-7765-4924-8047-ee4da4eb60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:08.820064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc46f8-7765-4924-8047-ee4da4eb60bf None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:08.820354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc46f8-7765-4924-8047-ee4da4eb60bf None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:08.820631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc46f8-7765-4924-8047-ee4da4eb60bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:08.821381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc46f8-7765-4924-8047-ee4da4eb60bf None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.domain.id=default, user.id=661f5d19be0f436ca14e6e9b440ca709) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:08.821696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc46f8-7765-4924-8047-ee4da4eb60bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:08.823210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc46f8-7765-4924-8047-ee4da4eb60bf None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:08.832507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 26/57] 10.222.0.71 () {56 vars in 1322 bytes} [Wed Sep 21 12:06:08 2022] GET /identity/v3/role_assignments?user.id=661f5d19be0f436ca14e6e9b440ca709&scope.domain.id=default&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 530 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:10.934299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089cc62-b417-43a7-8012-f5a3d39bd4aa None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:10.934791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089cc62-b417-43a7-8012-f5a3d39bd4aa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:10.934875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089cc62-b417-43a7-8012-f5a3d39bd4aa None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:10.936257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 32/58] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:10.941263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739967-d2a9-4224-a3e5-f8df3a0fa751 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:10.941624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739967-d2a9-4224-a3e5-f8df3a0fa751 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:10.941817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739967-d2a9-4224-a3e5-f8df3a0fa751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:10.959553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7e739967-d2a9-4224-a3e5-f8df3a0fa751 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:10.961092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7e739967-d2a9-4224-a3e5-f8df3a0fa751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:11.018292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:11.025717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 27/59] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:11.290971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089cc62-b417-43a7-8012-f5a3d39bd4aa None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:11.349740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:11.392154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089cc62-b417-43a7-8012-f5a3d39bd4aa None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:11.425770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469649e0-92af-4ccd-8958-34f0027787e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:11.426394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469649e0-92af-4ccd-8958-34f0027787e7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:11.426679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469649e0-92af-4ccd-8958-34f0027787e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:11.426959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469649e0-92af-4ccd-8958-34f0027787e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:11.428120 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469649e0-92af-4ccd-8958-34f0027787e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:11.429600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469649e0-92af-4ccd-8958-34f0027787e7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:11.431166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 33/60] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:11.437524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e739967-d2a9-4224-a3e5-f8df3a0fa751 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:11.533030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e739967-d2a9-4224-a3e5-f8df3a0fa751 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:11.565435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5c0db-6e65-470b-87cc-83b531c61499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:11.566042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5c0db-6e65-470b-87cc-83b531c61499 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:11.566324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5c0db-6e65-470b-87cc-83b531c61499 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:11.566597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5c0db-6e65-470b-87cc-83b531c61499 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:11.567242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5c0db-6e65-470b-87cc-83b531c61499 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:11.568716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5c0db-6e65-470b-87cc-83b531c61499 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:11.595954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 28/61] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:06:11 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:15.087699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204da0d3-baf0-4b91-83f2-1b048194ab31 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.088721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204da0d3-baf0-4b91-83f2-1b048194ab31 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.089792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204da0d3-baf0-4b91-83f2-1b048194ab31 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.091595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 34/62] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:15.097165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4717d-5cc0-4dbc-8fca-7b23b4d7b020 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.097992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4717d-5cc0-4dbc-8fca-7b23b4d7b020 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.098501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4717d-5cc0-4dbc-8fca-7b23b4d7b020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.124772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-abc4717d-5cc0-4dbc-8fca-7b23b4d7b020 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:15.126346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-abc4717d-5cc0-4dbc-8fca-7b23b4d7b020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:15.152174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d2fcb-2024-4eae-abe9-0967dc2017e1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.152473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d2fcb-2024-4eae-abe9-0967dc2017e1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.152922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d2fcb-2024-4eae-abe9-0967dc2017e1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.154279 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 35/63] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:15.159797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f128a9f-8abf-427c-9d00-8f64451921a0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.160355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f128a9f-8abf-427c-9d00-8f64451921a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.160827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f128a9f-8abf-427c-9d00-8f64451921a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.190695 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3f128a9f-8abf-427c-9d00-8f64451921a0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:15.191933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3f128a9f-8abf-427c-9d00-8f64451921a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:15.204126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:15.245225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 29/64] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:15.309104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 36/65] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:15.366616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a96b92-badf-4efb-a736-838c9c47b3a2 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.367214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a96b92-badf-4efb-a736-838c9c47b3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.367826 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a96b92-badf-4efb-a736-838c9c47b3a2 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.369401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 30/66] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:15.373966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48588357-aca1-41cf-962e-bd593c3e7fda None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.374260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48588357-aca1-41cf-962e-bd593c3e7fda None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.374544 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48588357-aca1-41cf-962e-bd593c3e7fda None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.376122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 37/67] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:15.380882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561abe7-608e-47a3-9d4e-c3dd657b94a6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.380882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561abe7-608e-47a3-9d4e-c3dd657b94a6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.380882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561abe7-608e-47a3-9d4e-c3dd657b94a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.389813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd4aac-0570-49ea-8279-733fd53c0a3c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.390084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd4aac-0570-49ea-8279-733fd53c0a3c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.390351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd4aac-0570-49ea-8279-733fd53c0a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.413735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1561abe7-608e-47a3-9d4e-c3dd657b94a6 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:15.414949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1561abe7-608e-47a3-9d4e-c3dd657b94a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:15.429790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-acfd4aac-0570-49ea-8279-733fd53c0a3c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:15.430990 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-acfd4aac-0570-49ea-8279-733fd53c0a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:15.538270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 31/68] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:15.547235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:15.577122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 38/69] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:15.590227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561abe7-608e-47a3-9d4e-c3dd657b94a6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:15.610814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd4aac-0570-49ea-8279-733fd53c0a3c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:15.718225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561abe7-608e-47a3-9d4e-c3dd657b94a6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:15.736657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:15.748189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd4aac-0570-49ea-8279-733fd53c0a3c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:15.771052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223817d-f102-4d57-b122-f4deaba18cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:15.771707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223817d-f102-4d57-b122-f4deaba18cff None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.772036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223817d-f102-4d57-b122-f4deaba18cff None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.772309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223817d-f102-4d57-b122-f4deaba18cff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.785399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223817d-f102-4d57-b122-f4deaba18cff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:15.788348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe77737-d221-449e-94ff-9e7b8e2ea5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:15.789156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe77737-d221-449e-94ff-9e7b8e2ea5fc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.789570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe77737-d221-449e-94ff-9e7b8e2ea5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.789978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe77737-d221-449e-94ff-9e7b8e2ea5fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3223817d-f102-4d57-b122-f4deaba18cff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:15.790435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:15.791420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:15.792599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 32/70] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:15.793316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ff643-04b6-43d5-9575-ba650e42d942 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.793590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ff643-04b6-43d5-9575-ba650e42d942 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.793857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ff643-04b6-43d5-9575-ba650e42d942 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.795097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 33/71] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:15.800066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ff643-04b6-43d5-9575-ba650e42d942 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:15.805416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe77737-d221-449e-94ff-9e7b8e2ea5fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2fe77737-d221-449e-94ff-9e7b8e2ea5fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:15.811163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:15.812019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:15.812780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:15.814982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 39/72] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:06:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:06:15.816916 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c766-a1cb-4bc4-ba20-6a07546a2b1b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.817427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c766-a1cb-4bc4-ba20-6a07546a2b1b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.817869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451c766-a1cb-4bc4-ba20-6a07546a2b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.837140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3451c766-a1cb-4bc4-ba20-6a07546a2b1b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:15.838735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3451c766-a1cb-4bc4-ba20-6a07546a2b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:15.885843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:15.911077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 40/73] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:15.911963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451c766-a1cb-4bc4-ba20-6a07546a2b1b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:15.913364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ff643-04b6-43d5-9575-ba650e42d942 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:15.951784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e49cde-8e7e-49e6-9ebf-d7cab33cc3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:15.952684 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e49cde-8e7e-49e6-9ebf-d7cab33cc3fe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:15.953045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e49cde-8e7e-49e6-9ebf-d7cab33cc3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:15.953322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e49cde-8e7e-49e6-9ebf-d7cab33cc3fe None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:15.953984 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e49cde-8e7e-49e6-9ebf-d7cab33cc3fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:15.954287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e49cde-8e7e-49e6-9ebf-d7cab33cc3fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:15.955681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e49cde-8e7e-49e6-9ebf-d7cab33cc3fe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:15.964519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 34/74] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:15.965594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e49cde-8e7e-49e6-9ebf-d7cab33cc3fe None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.000762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:16.029501 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451c766-a1cb-4bc4-ba20-6a07546a2b1b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.073151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36805ef4-5f08-4b95-b6b3-f755fd138420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.074333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36805ef4-5f08-4b95-b6b3-f755fd138420 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.076030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36805ef4-5f08-4b95-b6b3-f755fd138420 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.076316 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36805ef4-5f08-4b95-b6b3-f755fd138420 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.077145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36805ef4-5f08-4b95-b6b3-f755fd138420 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:16.077662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36805ef4-5f08-4b95-b6b3-f755fd138420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:16.079202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36805ef4-5f08-4b95-b6b3-f755fd138420 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:16.082902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e49cde-8e7e-49e6-9ebf-d7cab33cc3fe None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.095889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 41/75] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:16.095889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36805ef4-5f08-4b95-b6b3-f755fd138420 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.139683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:16.148967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3a246-ba07-4fd3-a129-0381f80b7d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.149607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3a246-ba07-4fd3-a129-0381f80b7d78 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.149889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3a246-ba07-4fd3-a129-0381f80b7d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.150172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3a246-ba07-4fd3-a129-0381f80b7d78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.151290 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3a246-ba07-4fd3-a129-0381f80b7d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:16.152814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3a246-ba07-4fd3-a129-0381f80b7d78 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:16.154394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 35/76] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:16.155158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3a246-ba07-4fd3-a129-0381f80b7d78 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.206799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36805ef4-5f08-4b95-b6b3-f755fd138420 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.242567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3b1ea-3867-431d-a004-b7403df28e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.243189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3b1ea-3867-431d-a004-b7403df28e0a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.243468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3b1ea-3867-431d-a004-b7403df28e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.243738 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3b1ea-3867-431d-a004-b7403df28e0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.254404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3b1ea-3867-431d-a004-b7403df28e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:16.256011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3a246-ba07-4fd3-a129-0381f80b7d78 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.257527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-89b3b1ea-3867-431d-a004-b7403df28e0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:16.259247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:16.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:16.260937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 42/77] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:06:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:16.261576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3b1ea-3867-431d-a004-b7403df28e0a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.299252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d4e06-2b98-413a-87d2-303ce4862c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.299925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4e06-2b98-413a-87d2-303ce4862c9f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.300211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4e06-2b98-413a-87d2-303ce4862c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.300487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4e06-2b98-413a-87d2-303ce4862c9f None admin] PATH_INFO: `/v3/users/2040a306bf664576a535972455e70a9a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.315932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d4e06-2b98-413a-87d2-303ce4862c9f None admin] RBAC: Authorizing `identity:get_user(user_id=2040a306bf664576a535972455e70a9a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:16.317405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d4e06-2b98-413a-87d2-303ce4862c9f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:16.319043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 36/78] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:16 2022] GET /identity/v3/users/2040a306bf664576a535972455e70a9a => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:16.319821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d4e06-2b98-413a-87d2-303ce4862c9f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.364073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:16.368460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3b1ea-3867-431d-a004-b7403df28e0a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.414547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76daf33c-d5f5-4345-bf82-f7254945ebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.415197 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daf33c-d5f5-4345-bf82-f7254945ebf1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.415481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daf33c-d5f5-4345-bf82-f7254945ebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.415774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daf33c-d5f5-4345-bf82-f7254945ebf1 None admin] PATH_INFO: `/v3/users/661f5d19be0f436ca14e6e9b440ca709` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.416944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76daf33c-d5f5-4345-bf82-f7254945ebf1 None admin] RBAC: Authorizing `identity:get_user(user_id=661f5d19be0f436ca14e6e9b440ca709)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:16.418409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76daf33c-d5f5-4345-bf82-f7254945ebf1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:16.420780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 43/79] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:16 2022] GET /identity/v3/users/661f5d19be0f436ca14e6e9b440ca709 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:16.421184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76daf33c-d5f5-4345-bf82-f7254945ebf1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.431413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d4e06-2b98-413a-87d2-303ce4862c9f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.481777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f106ae-083a-4ee4-af42-830b3c8254c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.482397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f106ae-083a-4ee4-af42-830b3c8254c3 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.482680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f106ae-083a-4ee4-af42-830b3c8254c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.482955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f106ae-083a-4ee4-af42-830b3c8254c3 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.483611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f106ae-083a-4ee4-af42-830b3c8254c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:16.485103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f106ae-083a-4ee4-af42-830b3c8254c3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:16.505812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 37/80] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:06:16 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:16.508747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f106ae-083a-4ee4-af42-830b3c8254c3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.533732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:16.538278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76daf33c-d5f5-4345-bf82-f7254945ebf1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.577068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fb056-91e9-47a3-9337-4168b38dd3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.577717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fb056-91e9-47a3-9337-4168b38dd3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.578066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fb056-91e9-47a3-9337-4168b38dd3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.578335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fb056-91e9-47a3-9337-4168b38dd3a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.590493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fb056-91e9-47a3-9337-4168b38dd3a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7a8fb056-91e9-47a3-9337-4168b38dd3a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:16.595565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:16.596700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:16.597447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 44/81] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:16.598067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fb056-91e9-47a3-9337-4168b38dd3a7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.627524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:16.643929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f106ae-083a-4ee4-af42-830b3c8254c3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.683503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066458d-975f-475f-9340-c972dddb101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.684162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066458d-975f-475f-9340-c972dddb101c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.684448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066458d-975f-475f-9340-c972dddb101c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.684724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066458d-975f-475f-9340-c972dddb101c None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.685395 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066458d-975f-475f-9340-c972dddb101c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:16.685705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066458d-975f-475f-9340-c972dddb101c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:16.687108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066458d-975f-475f-9340-c972dddb101c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:16.696416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 38/82] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:06:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:16.697264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066458d-975f-475f-9340-c972dddb101c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.713786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fb056-91e9-47a3-9337-4168b38dd3a7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.758772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d41da-f5e1-42d4-aff5-8a916d2c692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.759858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d41da-f5e1-42d4-aff5-8a916d2c692c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.760361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d41da-f5e1-42d4-aff5-8a916d2c692c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.760827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d41da-f5e1-42d4-aff5-8a916d2c692c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.817250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:16.842974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066458d-975f-475f-9340-c972dddb101c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:16.855711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d41da-f5e1-42d4-aff5-8a916d2c692c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:16.857249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d41da-f5e1-42d4-aff5-8a916d2c692c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:16.887352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a75183-c5da-4ece-9753-1f5282784fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:16.888241 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a75183-c5da-4ece-9753-1f5282784fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:16.888647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a75183-c5da-4ece-9753-1f5282784fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:16.889049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a75183-c5da-4ece-9753-1f5282784fe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:16.890103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:16.968910 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 45/83] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:16.969580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d41da-f5e1-42d4-aff5-8a916d2c692c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:16.974963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a75183-c5da-4ece-9753-1f5282784fe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:16.976486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a75183-c5da-4ece-9753-1f5282784fe0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:17.077185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:17.079644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d41da-f5e1-42d4-aff5-8a916d2c692c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:17.106505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 39/84] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:17.107969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:17.108397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a75183-c5da-4ece-9753-1f5282784fe0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:17.126032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b2948-9e84-41cb-9632-e30893060ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:17.126709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b2948-9e84-41cb-9632-e30893060ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:17.126998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b2948-9e84-41cb-9632-e30893060ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:17.127276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b2948-9e84-41cb-9632-e30893060ed8 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:17.127991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b2948-9e84-41cb-9632-e30893060ed8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:17.128306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b2948-9e84-41cb-9632-e30893060ed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:17.129955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b2948-9e84-41cb-9632-e30893060ed8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:17.141830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 46/85] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:17.142914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b2948-9e84-41cb-9632-e30893060ed8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:17.228275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a75183-c5da-4ece-9753-1f5282784fe0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:17.265742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b2948-9e84-41cb-9632-e30893060ed8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:17.279449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f053d91-e567-4ff4-ad96-00fc52f50446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:17.280145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f053d91-e567-4ff4-ad96-00fc52f50446 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:17.280685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f053d91-e567-4ff4-ad96-00fc52f50446 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:17.280972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f053d91-e567-4ff4-ad96-00fc52f50446 None admin] PATH_INFO: `/v3/users/2040a306bf664576a535972455e70a9a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:17.282637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f053d91-e567-4ff4-ad96-00fc52f50446 None admin] RBAC: Authorizing `identity:get_user(user_id=2040a306bf664576a535972455e70a9a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:17.284422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f053d91-e567-4ff4-ad96-00fc52f50446 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:17.286731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 40/86] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:17 2022] GET /identity/v3/users/2040a306bf664576a535972455e70a9a => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:17.292741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f053d91-e567-4ff4-ad96-00fc52f50446 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:17.306572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed42dd-e3bd-434a-9b33-700579e591e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:17.307666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed42dd-e3bd-434a-9b33-700579e591e5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:17.308713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed42dd-e3bd-434a-9b33-700579e591e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:17.309230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed42dd-e3bd-434a-9b33-700579e591e5 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:17.310639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed42dd-e3bd-434a-9b33-700579e591e5 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:17.312539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed42dd-e3bd-434a-9b33-700579e591e5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:17.314557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 47/87] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:17 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:17.315674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed42dd-e3bd-434a-9b33-700579e591e5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:17.357145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:17.387285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:17.421039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f053d91-e567-4ff4-ad96-00fc52f50446 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:17.443107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed42dd-e3bd-434a-9b33-700579e591e5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:17.459446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccba42-806c-4d46-95d2-c55f05923a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:17.460583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccba42-806c-4d46-95d2-c55f05923a1d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:17.461174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccba42-806c-4d46-95d2-c55f05923a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:17.461639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccba42-806c-4d46-95d2-c55f05923a1d None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:17.463211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccba42-806c-4d46-95d2-c55f05923a1d None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:17.465981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccba42-806c-4d46-95d2-c55f05923a1d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:17.468139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 41/88] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:17 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:17.469259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccba42-806c-4d46-95d2-c55f05923a1d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:17.492380 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183494c-3b1d-45e3-b299-016e59125ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:17.493029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183494c-3b1d-45e3-b299-016e59125ece None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:17.493317 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183494c-3b1d-45e3-b299-016e59125ece None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:17.493600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183494c-3b1d-45e3-b299-016e59125ece None admin] PATH_INFO: `/v3/users/2040a306bf664576a535972455e70a9a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:17.494779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183494c-3b1d-45e3-b299-016e59125ece None admin] RBAC: Authorizing `identity:get_user(user_id=2040a306bf664576a535972455e70a9a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:17.496878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183494c-3b1d-45e3-b299-016e59125ece None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:17.498803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 48/89] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:17 2022] GET /identity/v3/users/2040a306bf664576a535972455e70a9a => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:17.503493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183494c-3b1d-45e3-b299-016e59125ece None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:17.600479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccba42-806c-4d46-95d2-c55f05923a1d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:17.621332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:17.630912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183494c-3b1d-45e3-b299-016e59125ece None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:17.647452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd9f51-9019-4e1c-9e79-6c853e196777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:17.648350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd9f51-9019-4e1c-9e79-6c853e196777 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:17.648658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd9f51-9019-4e1c-9e79-6c853e196777 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:17.648939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fd9f51-9019-4e1c-9e79-6c853e196777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:17.657567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:17.671939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539804b3-5d08-4f23-8a53-ea5df94e7a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:17.672921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539804b3-5d08-4f23-8a53-ea5df94e7a6a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:17.673396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539804b3-5d08-4f23-8a53-ea5df94e7a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:17.673928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539804b3-5d08-4f23-8a53-ea5df94e7a6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:17.674840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539804b3-5d08-4f23-8a53-ea5df94e7a6a None admin] RBAC: Adding query filter params (role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=811b356a854c422f85cddd17b5c478aa, user.id=2040a306bf664576a535972455e70a9a) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:17.675468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539804b3-5d08-4f23-8a53-ea5df94e7a6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:17.677136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539804b3-5d08-4f23-8a53-ea5df94e7a6a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:17.684024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 49/90] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:17 2022] GET /identity/v3/role_assignments?user.id=2040a306bf664576a535972455e70a9a&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:17.685101 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539804b3-5d08-4f23-8a53-ea5df94e7a6a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:17.739073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd9f51-9019-4e1c-9e79-6c853e196777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:17.740867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fd9f51-9019-4e1c-9e79-6c853e196777 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:17.826648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539804b3-5d08-4f23-8a53-ea5df94e7a6a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:17.919617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa7ee0-0682-4fa1-abe8-d4e067a8fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:17.920187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa7ee0-0682-4fa1-abe8-d4e067a8fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:17.920472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa7ee0-0682-4fa1-abe8-d4e067a8fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:17.920750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa7ee0-0682-4fa1-abe8-d4e067a8fdd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:17.921493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa7ee0-0682-4fa1-abe8-d4e067a8fdd6 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=811b356a854c422f85cddd17b5c478aa, user.id=661f5d19be0f436ca14e6e9b440ca709) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:17.921807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa7ee0-0682-4fa1-abe8-d4e067a8fdd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:17.923264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa7ee0-0682-4fa1-abe8-d4e067a8fdd6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:17.930770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 50/91] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:17 2022] GET /identity/v3/role_assignments?user.id=661f5d19be0f436ca14e6e9b440ca709&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:17.932772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa7ee0-0682-4fa1-abe8-d4e067a8fdd6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:17.943632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 42/92] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:17.949995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd9f51-9019-4e1c-9e79-6c853e196777 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:17.983430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:18.051726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:18.144978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa7ee0-0682-4fa1-abe8-d4e067a8fdd6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:18.185403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fd9f51-9019-4e1c-9e79-6c853e196777 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:18.207328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e3aeb-b4aa-4596-8a7b-d2ea130d15ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:18.208483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e3aeb-b4aa-4596-8a7b-d2ea130d15ca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:18.208999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e3aeb-b4aa-4596-8a7b-d2ea130d15ca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:18.209462 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e3aeb-b4aa-4596-8a7b-d2ea130d15ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:18.238097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2be392-1a97-46ca-8b0e-6668d20e81f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:18.238761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2be392-1a97-46ca-8b0e-6668d20e81f0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:18.239059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2be392-1a97-46ca-8b0e-6668d20e81f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:18.239345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2be392-1a97-46ca-8b0e-6668d20e81f0 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:18.241550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2be392-1a97-46ca-8b0e-6668d20e81f0 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:18.243180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2be392-1a97-46ca-8b0e-6668d20e81f0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:18.246138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 43/93] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:17 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:18.251075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2be392-1a97-46ca-8b0e-6668d20e81f0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:18.321662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e3aeb-b4aa-4596-8a7b-d2ea130d15ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:18.324387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e3aeb-b4aa-4596-8a7b-d2ea130d15ca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:18.349010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:18.388972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2be392-1a97-46ca-8b0e-6668d20e81f0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:18.413122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:18.461565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065b983-1cc7-4754-8bf7-424a90396755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:18.462518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065b983-1cc7-4754-8bf7-424a90396755 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:18.463007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065b983-1cc7-4754-8bf7-424a90396755 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:18.463523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065b983-1cc7-4754-8bf7-424a90396755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:18.465500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065b983-1cc7-4754-8bf7-424a90396755 None admin] RBAC: Adding query filter params (role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=811b356a854c422f85cddd17b5c478aa, user.id=2040a306bf664576a535972455e70a9a) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:18.466223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065b983-1cc7-4754-8bf7-424a90396755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:18.469552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065b983-1cc7-4754-8bf7-424a90396755 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:18.479723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 44/94] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:18 2022] GET /identity/v3/role_assignments?user.id=2040a306bf664576a535972455e70a9a&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:18.512196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 51/95] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:18.517948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065b983-1cc7-4754-8bf7-424a90396755 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:18.649652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065b983-1cc7-4754-8bf7-424a90396755 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:18.700590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12af316-b22b-4e5a-9423-4779a7c6875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:18.701695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12af316-b22b-4e5a-9423-4779a7c6875f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:18.702223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12af316-b22b-4e5a-9423-4779a7c6875f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:18.702689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12af316-b22b-4e5a-9423-4779a7c6875f None admin] PATH_INFO: `/v3/projects/8f4741383fee4714834b4003f173590b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:18.704124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12af316-b22b-4e5a-9423-4779a7c6875f None admin] RBAC: Authorizing `identity:get_project(project_id=8f4741383fee4714834b4003f173590b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:18.705791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12af316-b22b-4e5a-9423-4779a7c6875f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:18.707675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 45/96] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:18 2022] GET /identity/v3/projects/8f4741383fee4714834b4003f173590b => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:18.713945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e3aeb-b4aa-4596-8a7b-d2ea130d15ca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:18.860348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e3aeb-b4aa-4596-8a7b-d2ea130d15ca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:18.904504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ae322-ba99-42cb-b1a2-b52baf25fb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:18.905182 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae322-ba99-42cb-b1a2-b52baf25fb3b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:18.905467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae322-ba99-42cb-b1a2-b52baf25fb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:18.905739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae322-ba99-42cb-b1a2-b52baf25fb3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:18.906483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ae322-ba99-42cb-b1a2-b52baf25fb3b None admin] RBAC: Adding query filter params (role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=8f4741383fee4714834b4003f173590b, user.id=2040a306bf664576a535972455e70a9a) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:18.906794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ae322-ba99-42cb-b1a2-b52baf25fb3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:18.908281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600ae322-ba99-42cb-b1a2-b52baf25fb3b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:18.914947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 52/97] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:18 2022] GET /identity/v3/role_assignments?user.id=2040a306bf664576a535972455e70a9a&scope.project.id=8f4741383fee4714834b4003f173590b&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:19.210715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ad2c5-c2ae-4226-a6c6-c252d7181756 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:19.211138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ad2c5-c2ae-4226-a6c6-c252d7181756 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:19.211340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ad2c5-c2ae-4226-a6c6-c252d7181756 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:19.212813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 46/98] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:19.220554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b20636-db7e-43b9-87ed-0c4585791917 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:19.221045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b20636-db7e-43b9-87ed-0c4585791917 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:19.221143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b20636-db7e-43b9-87ed-0c4585791917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:19.243425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-49b20636-db7e-43b9-87ed-0c4585791917 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:19.244972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-49b20636-db7e-43b9-87ed-0c4585791917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:19.263048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:19.267679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d793e53-b6a9-4289-be60-7e38afb1963b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:19.268563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d793e53-b6a9-4289-be60-7e38afb1963b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:19.269057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d793e53-b6a9-4289-be60-7e38afb1963b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:19.272017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 47/99] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:19.277746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0bfb2-3672-4a90-b0b1-0ba128f4460d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:19.278306 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0bfb2-3672-4a90-b0b1-0ba128f4460d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:19.278771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0bfb2-3672-4a90-b0b1-0ba128f4460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:19.309625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-31a0bfb2-3672-4a90-b0b1-0ba128f4460d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:19.310865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-31a0bfb2-3672-4a90-b0b1-0ba128f4460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:19.331123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:19.378406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 53/100] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:19.436711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 48/101] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:19.825393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b20636-db7e-43b9-87ed-0c4585791917 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:19.944335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b20636-db7e-43b9-87ed-0c4585791917 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:19.955543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0bfb2-3672-4a90-b0b1-0ba128f4460d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:19.993943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494abe0f-6266-4fe1-920d-162be8116481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:19.994594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494abe0f-6266-4fe1-920d-162be8116481 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:19.994878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494abe0f-6266-4fe1-920d-162be8116481 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:19.995156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494abe0f-6266-4fe1-920d-162be8116481 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:20.006595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494abe0f-6266-4fe1-920d-162be8116481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-494abe0f-6266-4fe1-920d-162be8116481 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:20.011550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:20.012558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:20.013352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 54/102] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:20.018755 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494abe0f-6266-4fe1-920d-162be8116481 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:20.028142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:20.180060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0bfb2-3672-4a90-b0b1-0ba128f4460d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:20.198285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494abe0f-6266-4fe1-920d-162be8116481 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:20.263577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3885a3-5a21-402d-af4a-4f04ae27c529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:20.264486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3885a3-5a21-402d-af4a-4f04ae27c529 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:20.266452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3885a3-5a21-402d-af4a-4f04ae27c529 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:20.267499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3885a3-5a21-402d-af4a-4f04ae27c529 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:20.269502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3885a3-5a21-402d-af4a-4f04ae27c529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:20.272113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3885a3-5a21-402d-af4a-4f04ae27c529 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:20.274024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 49/103] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:20.278643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded05a6-fea7-4816-b5e4-65e7dfd94a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:20.279744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded05a6-fea7-4816-b5e4-65e7dfd94a63 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:20.280199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded05a6-fea7-4816-b5e4-65e7dfd94a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:20.280630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded05a6-fea7-4816-b5e4-65e7dfd94a63 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:20.281448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded05a6-fea7-4816-b5e4-65e7dfd94a63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:20.282856 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3885a3-5a21-402d-af4a-4f04ae27c529 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:20.283885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded05a6-fea7-4816-b5e4-65e7dfd94a63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:20.288111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded05a6-fea7-4816-b5e4-65e7dfd94a63 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:20.305336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 55/104] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:20.312529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:20.313698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded05a6-fea7-4816-b5e4-65e7dfd94a63 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:20.462183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:20.512795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3885a3-5a21-402d-af4a-4f04ae27c529 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:20.545466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded05a6-fea7-4816-b5e4-65e7dfd94a63 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:20.607794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b1df8-367a-478a-8478-8a72334e93f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:20.608864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b1df8-367a-478a-8478-8a72334e93f0 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:20.609369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b1df8-367a-478a-8478-8a72334e93f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:20.609877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b1df8-367a-478a-8478-8a72334e93f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:20.611079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b1df8-367a-478a-8478-8a72334e93f0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:20.613857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b1df8-367a-478a-8478-8a72334e93f0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:20.633651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f919cd1-e109-4f86-8fa8-d5f238c5445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:20.634727 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f919cd1-e109-4f86-8fa8-d5f238c5445a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:20.635296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f919cd1-e109-4f86-8fa8-d5f238c5445a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:20.635891 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f919cd1-e109-4f86-8fa8-d5f238c5445a None admin] PATH_INFO: `/v3/users/2cde2e98da504619a842cc2feb481e74` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:20.672940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f919cd1-e109-4f86-8fa8-d5f238c5445a None admin] RBAC: Authorizing `identity:get_user(user_id=2cde2e98da504619a842cc2feb481e74)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:20.675939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f919cd1-e109-4f86-8fa8-d5f238c5445a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:20.679345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 56/105] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:20 2022] GET /identity/v3/users/2cde2e98da504619a842cc2feb481e74 => generated 289 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:20.689797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f919cd1-e109-4f86-8fa8-d5f238c5445a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:20.704861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 50/106] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:06:20 2022] POST /identity/v3/projects => generated 302 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:20.866171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f919cd1-e109-4f86-8fa8-d5f238c5445a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:20.902939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:20.912408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50023cf-83e1-4f0b-b71b-045eb31bea14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:20.913909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50023cf-83e1-4f0b-b71b-045eb31bea14 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:20.914585 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50023cf-83e1-4f0b-b71b-045eb31bea14 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:20.915363 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50023cf-83e1-4f0b-b71b-045eb31bea14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:21.009560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50023cf-83e1-4f0b-b71b-045eb31bea14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:21.011117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50023cf-83e1-4f0b-b71b-045eb31bea14 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:21.160218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 57/107] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 12:06:21.165902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b1df8-367a-478a-8478-8a72334e93f0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:21.247002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:21.297476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b1df8-367a-478a-8478-8a72334e93f0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:21.342070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebeb095-0e4b-4919-9e4e-1c57fbb8ac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:21.343091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeb095-0e4b-4919-9e4e-1c57fbb8ac9c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:21.343597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeb095-0e4b-4919-9e4e-1c57fbb8ac9c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:21.344118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeb095-0e4b-4919-9e4e-1c57fbb8ac9c None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:21.345706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebeb095-0e4b-4919-9e4e-1c57fbb8ac9c None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:21.347536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebeb095-0e4b-4919-9e4e-1c57fbb8ac9c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:21.349602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 51/108] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:21 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:21.356397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50023cf-83e1-4f0b-b71b-045eb31bea14 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:21.379773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b689e-db62-4f08-bc7d-2bd6b5e31581 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:21.380105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b689e-db62-4f08-bc7d-2bd6b5e31581 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:21.380387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b689e-db62-4f08-bc7d-2bd6b5e31581 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:21.381872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 52/109] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:21.387426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b1846-af8d-47ed-866e-ec907abd32b4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:21.387885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b1846-af8d-47ed-866e-ec907abd32b4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:21.388059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b1846-af8d-47ed-866e-ec907abd32b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:21.414050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-485b1846-af8d-47ed-866e-ec907abd32b4 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:21.415173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-485b1846-af8d-47ed-866e-ec907abd32b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:21.422491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:21.575729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 53/110] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:21.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50023cf-83e1-4f0b-b71b-045eb31bea14 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:21.625721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37b301-ba59-4229-aa89-1cc18cd52a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:21.626593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b301-ba59-4229-aa89-1cc18cd52a49 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:21.626884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b301-ba59-4229-aa89-1cc18cd52a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:21.627164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37b301-ba59-4229-aa89-1cc18cd52a49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:21.627937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37b301-ba59-4229-aa89-1cc18cd52a49 None admin] RBAC: Adding query filter params (role.id=9dbcf5e8a10a4728888b9f4e923f5814, scope.project.id=811b356a854c422f85cddd17b5c478aa, user.id=2cde2e98da504619a842cc2feb481e74) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:21.628252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37b301-ba59-4229-aa89-1cc18cd52a49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:21.629761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f37b301-ba59-4229-aa89-1cc18cd52a49 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:21.636492 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 58/111] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:21 2022] GET /identity/v3/role_assignments?user.id=2cde2e98da504619a842cc2feb481e74&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=9dbcf5e8a10a4728888b9f4e923f5814 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:21.788465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e466-6ab6-488d-9658-5992c1691539 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:21.788806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e466-6ab6-488d-9658-5992c1691539 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:21.789037 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1e466-6ab6-488d-9658-5992c1691539 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:21.790657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 54/112] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:21.797003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea0f2-e1c0-4fb5-b9f5-a36aa90a8849 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:21.797338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea0f2-e1c0-4fb5-b9f5-a36aa90a8849 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:21.797568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ea0f2-e1c0-4fb5-b9f5-a36aa90a8849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:21.824788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-112ea0f2-e1c0-4fb5-b9f5-a36aa90a8849 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:21.826117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-112ea0f2-e1c0-4fb5-b9f5-a36aa90a8849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:21.912030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 59/113] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:21.918593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e7c6d-5db2-487b-a68c-7df53a374378 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:21.918884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e7c6d-5db2-487b-a68c-7df53a374378 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:21.919165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e7c6d-5db2-487b-a68c-7df53a374378 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:21.920563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 55/114] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:21.926061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed64b36-8aea-40ca-bd5d-47cd339f2833 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:21.926625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed64b36-8aea-40ca-bd5d-47cd339f2833 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:21.927110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed64b36-8aea-40ca-bd5d-47cd339f2833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:21.952200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5ed64b36-8aea-40ca-bd5d-47cd339f2833 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:21.953748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5ed64b36-8aea-40ca-bd5d-47cd339f2833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:21.976868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:21.990375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e7c6d-5db2-487b-a68c-7df53a374378 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:22.068245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 60/115] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:22.088134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:22.132388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e7c6d-5db2-487b-a68c-7df53a374378 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:22.177714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ff15c-98e5-4ed8-bad5-3e288e888610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:22.178419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff15c-98e5-4ed8-bad5-3e288e888610 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:22.178703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff15c-98e5-4ed8-bad5-3e288e888610 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:22.178979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ff15c-98e5-4ed8-bad5-3e288e888610 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:22.190736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ff15c-98e5-4ed8-bad5-3e288e888610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-693ff15c-98e5-4ed8-bad5-3e288e888610 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:22.195811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:22.196669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:22.197407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:22.202325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 56/116] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:06:22.203689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed64b36-8aea-40ca-bd5d-47cd339f2833 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:22.266505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ff15c-98e5-4ed8-bad5-3e288e888610 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:22.379013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed64b36-8aea-40ca-bd5d-47cd339f2833 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:22.423960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51672e0f-b965-46c5-bd7c-04489ab93a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:22.424583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51672e0f-b965-46c5-bd7c-04489ab93a79 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:22.424871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51672e0f-b965-46c5-bd7c-04489ab93a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:22.425147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51672e0f-b965-46c5-bd7c-04489ab93a79 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:22.425829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51672e0f-b965-46c5-bd7c-04489ab93a79 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:22.426137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51672e0f-b965-46c5-bd7c-04489ab93a79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:22.427662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51672e0f-b965-46c5-bd7c-04489ab93a79 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:22.437900 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 61/117] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:22.443184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51672e0f-b965-46c5-bd7c-04489ab93a79 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:22.458350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ff15c-98e5-4ed8-bad5-3e288e888610 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:22.472592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:22.512136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:22.530539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede9566-4c9b-4343-8ea8-6e6aa162d2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:22.531185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede9566-4c9b-4343-8ea8-6e6aa162d2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:22.531463 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede9566-4c9b-4343-8ea8-6e6aa162d2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:22.531948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede9566-4c9b-4343-8ea8-6e6aa162d2e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:22.553060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede9566-4c9b-4343-8ea8-6e6aa162d2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-aede9566-4c9b-4343-8ea8-6e6aa162d2e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:22.560645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:22.561561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:22.562319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 57/118] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:06:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:22.562962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede9566-4c9b-4343-8ea8-6e6aa162d2e5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:22.629233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51672e0f-b965-46c5-bd7c-04489ab93a79 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:22.671577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d18fc-550b-4f48-b190-cf55c4dc836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:22.672260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d18fc-550b-4f48-b190-cf55c4dc836e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:22.672589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d18fc-550b-4f48-b190-cf55c4dc836e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:22.672863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90d18fc-550b-4f48-b190-cf55c4dc836e None admin] PATH_INFO: `/v3/users/2040a306bf664576a535972455e70a9a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:22.674033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d18fc-550b-4f48-b190-cf55c4dc836e None admin] RBAC: Authorizing `identity:get_user(user_id=2040a306bf664576a535972455e70a9a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:22.676060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90d18fc-550b-4f48-b190-cf55c4dc836e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:22.680183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 62/119] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:22 2022] GET /identity/v3/users/2040a306bf664576a535972455e70a9a => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:22.681106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d18fc-550b-4f48-b190-cf55c4dc836e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:22.707876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede9566-4c9b-4343-8ea8-6e6aa162d2e5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:22.750227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81853d03-25da-44aa-a46e-81b004bd1007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:22.750877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81853d03-25da-44aa-a46e-81b004bd1007 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:22.751162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81853d03-25da-44aa-a46e-81b004bd1007 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:22.751440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81853d03-25da-44aa-a46e-81b004bd1007 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:22.768961 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81853d03-25da-44aa-a46e-81b004bd1007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-81853d03-25da-44aa-a46e-81b004bd1007 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:22.774227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:22.775266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:22.776168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:22.777959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 58/120] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:06:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:22.779284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7701985-eb27-42a9-82cb-6c9c4bf9cd5c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:22.779832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7701985-eb27-42a9-82cb-6c9c4bf9cd5c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:22.780255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7701985-eb27-42a9-82cb-6c9c4bf9cd5c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:22.781697 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 59/121] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:22.782635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7701985-eb27-42a9-82cb-6c9c4bf9cd5c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:22.795968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:22.836227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90d18fc-550b-4f48-b190-cf55c4dc836e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:22.845740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:22.883998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72380732-dee7-450e-aafb-fd5c71bd9902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:22.884757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72380732-dee7-450e-aafb-fd5c71bd9902 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:22.885118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72380732-dee7-450e-aafb-fd5c71bd9902 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:22.885552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72380732-dee7-450e-aafb-fd5c71bd9902 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:22.887141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72380732-dee7-450e-aafb-fd5c71bd9902 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:22.887330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72380732-dee7-450e-aafb-fd5c71bd9902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:22.888980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72380732-dee7-450e-aafb-fd5c71bd9902 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:22.899022 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 63/122] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:22.904733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72380732-dee7-450e-aafb-fd5c71bd9902 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:22.930458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7701985-eb27-42a9-82cb-6c9c4bf9cd5c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:22.979545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab78f0-60e3-435d-9cad-1092465c9b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:22.981160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab78f0-60e3-435d-9cad-1092465c9b8b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:22.981656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab78f0-60e3-435d-9cad-1092465c9b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:22.982327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab78f0-60e3-435d-9cad-1092465c9b8b None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:22.984287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab78f0-60e3-435d-9cad-1092465c9b8b None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:22.986543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab78f0-60e3-435d-9cad-1092465c9b8b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:22.989371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 60/123] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:22 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:22.990375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427219cf-dcb2-46aa-b1e9-18f68c15797e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:22.990663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427219cf-dcb2-46aa-b1e9-18f68c15797e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:22.990939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427219cf-dcb2-46aa-b1e9-18f68c15797e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:23.009911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-427219cf-dcb2-46aa-b1e9-18f68c15797e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:23.011200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-427219cf-dcb2-46aa-b1e9-18f68c15797e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:23.030947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72380732-dee7-450e-aafb-fd5c71bd9902 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:23.082163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4de8f6-3443-4c29-8ce0-f86d1a736a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:23.083398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4de8f6-3443-4c29-8ce0-f86d1a736a79 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:23.083951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4de8f6-3443-4c29-8ce0-f86d1a736a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:23.084481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4de8f6-3443-4c29-8ce0-f86d1a736a79 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:23.085373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4de8f6-3443-4c29-8ce0-f86d1a736a79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:23.085825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4de8f6-3443-4c29-8ce0-f86d1a736a79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:23.087037 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 61/124] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:23.087529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4de8f6-3443-4c29-8ce0-f86d1a736a79 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:23.088245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427219cf-dcb2-46aa-b1e9-18f68c15797e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:23.099504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 64/125] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:06:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:23.100759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4de8f6-3443-4c29-8ce0-f86d1a736a79 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:23.106641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:23.154521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:23.265601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427219cf-dcb2-46aa-b1e9-18f68c15797e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:23.311492 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4de8f6-3443-4c29-8ce0-f86d1a736a79 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:23.336519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d78bc1-73ae-475d-bc52-c7876a460c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:23.337168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d78bc1-73ae-475d-bc52-c7876a460c80 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:23.337443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d78bc1-73ae-475d-bc52-c7876a460c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:23.337712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d78bc1-73ae-475d-bc52-c7876a460c80 None admin] PATH_INFO: `/v3/users/661f5d19be0f436ca14e6e9b440ca709` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:23.340713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d78bc1-73ae-475d-bc52-c7876a460c80 None admin] RBAC: Authorizing `identity:get_user(user_id=661f5d19be0f436ca14e6e9b440ca709)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:23.342177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d78bc1-73ae-475d-bc52-c7876a460c80 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:23.343843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 62/126] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:23 2022] GET /identity/v3/users/661f5d19be0f436ca14e6e9b440ca709 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:23.344621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d78bc1-73ae-475d-bc52-c7876a460c80 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:23.368254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775e0e3-522a-4865-b79e-ff848b57f632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:23.368920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775e0e3-522a-4865-b79e-ff848b57f632 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:23.369198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775e0e3-522a-4865-b79e-ff848b57f632 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:23.369471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775e0e3-522a-4865-b79e-ff848b57f632 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa/users/2040a306bf664576a535972455e70a9a/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:23.371579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775e0e3-522a-4865-b79e-ff848b57f632 None admin] RBAC: Authorizing `identity:create_grant(project_id=811b356a854c422f85cddd17b5c478aa, user_id=2040a306bf664576a535972455e70a9a, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:23.373061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775e0e3-522a-4865-b79e-ff848b57f632 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:23.390932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 65/127] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:06:23 2022] PUT /identity/v3/projects/811b356a854c422f85cddd17b5c478aa/users/2040a306bf664576a535972455e70a9a/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:23.400979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775e0e3-522a-4865-b79e-ff848b57f632 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:23.477781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d78bc1-73ae-475d-bc52-c7876a460c80 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:23.490147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:23.519767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c325-4937-4960-acbd-61b1bc388e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:23.520440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c325-4937-4960-acbd-61b1bc388e47 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:23.520720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c325-4937-4960-acbd-61b1bc388e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:23.520992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c325-4937-4960-acbd-61b1bc388e47 None admin] PATH_INFO: `/v3/users/2040a306bf664576a535972455e70a9a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:23.522395 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951c325-4937-4960-acbd-61b1bc388e47 None admin] RBAC: Authorizing `identity:get_user(user_id=2040a306bf664576a535972455e70a9a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:23.523933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951c325-4937-4960-acbd-61b1bc388e47 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:23.525917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 63/128] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:23 2022] GET /identity/v3/users/2040a306bf664576a535972455e70a9a => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:23.527317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c325-4937-4960-acbd-61b1bc388e47 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:23.535253 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:23.619288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775e0e3-522a-4865-b79e-ff848b57f632 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:23.645367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c325-4937-4960-acbd-61b1bc388e47 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:23.683052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368f4f7-b50a-463b-8475-4f4b31d6f6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:23.683704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368f4f7-b50a-463b-8475-4f4b31d6f6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:23.684019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368f4f7-b50a-463b-8475-4f4b31d6f6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:23.684291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368f4f7-b50a-463b-8475-4f4b31d6f6a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:23.696092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368f4f7-b50a-463b-8475-4f4b31d6f6a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:23.700541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb0fff-6e44-4368-930c-cceb13ec3097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:23.701199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb0fff-6e44-4368-930c-cceb13ec3097 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:23.701480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb0fff-6e44-4368-930c-cceb13ec3097 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a368f4f7-b50a-463b-8475-4f4b31d6f6a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:23.701582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:23.702474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:23.703242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:23.703868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb0fff-6e44-4368-930c-cceb13ec3097 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:23.703950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 64/129] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:23.703950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368f4f7-b50a-463b-8475-4f4b31d6f6a9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:23.707779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb0fff-6e44-4368-930c-cceb13ec3097 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:23.707779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb0fff-6e44-4368-930c-cceb13ec3097 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:23.711998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 66/130] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:23 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:23.714910 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb0fff-6e44-4368-930c-cceb13ec3097 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:23.751879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:23.822171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368f4f7-b50a-463b-8475-4f4b31d6f6a9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:23.858637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa020a-f39a-4e3f-9c41-608832c65708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:23.859282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa020a-f39a-4e3f-9c41-608832c65708 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:23.859561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa020a-f39a-4e3f-9c41-608832c65708 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:23.859862 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa020a-f39a-4e3f-9c41-608832c65708 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:23.861049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa020a-f39a-4e3f-9c41-608832c65708 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:23.862496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa020a-f39a-4e3f-9c41-608832c65708 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:23.864149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 65/131] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:23 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:23.864936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa020a-f39a-4e3f-9c41-608832c65708 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:23.888580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb0fff-6e44-4368-930c-cceb13ec3097 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:23.926735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:23.933911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a1c78-5bd7-415a-85ec-c7b8bfc7e3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:23.934926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a1c78-5bd7-415a-85ec-c7b8bfc7e3d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:23.935277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a1c78-5bd7-415a-85ec-c7b8bfc7e3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:23.935626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a1c78-5bd7-415a-85ec-c7b8bfc7e3d7 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa/users/661f5d19be0f436ca14e6e9b440ca709/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:23.938451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a1c78-5bd7-415a-85ec-c7b8bfc7e3d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=811b356a854c422f85cddd17b5c478aa, user_id=661f5d19be0f436ca14e6e9b440ca709, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:23.940186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a1c78-5bd7-415a-85ec-c7b8bfc7e3d7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:23.960192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 67/132] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:06:23 2022] PUT /identity/v3/projects/811b356a854c422f85cddd17b5c478aa/users/661f5d19be0f436ca14e6e9b440ca709/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:06:23.961476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a1c78-5bd7-415a-85ec-c7b8bfc7e3d7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:23.988029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa020a-f39a-4e3f-9c41-608832c65708 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:23.991300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:24.030703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be01b8-fb61-45aa-bb81-f8fd8236d22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:24.031242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be01b8-fb61-45aa-bb81-f8fd8236d22d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:24.031568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be01b8-fb61-45aa-bb81-f8fd8236d22d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:24.031890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be01b8-fb61-45aa-bb81-f8fd8236d22d None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:24.032564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be01b8-fb61-45aa-bb81-f8fd8236d22d None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:24.032918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be01b8-fb61-45aa-bb81-f8fd8236d22d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:24.034422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be01b8-fb61-45aa-bb81-f8fd8236d22d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:24.043478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 66/133] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:24.044770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d72f95-aa99-4fda-8a1f-e1fbfd216900 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:24.049820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d72f95-aa99-4fda-8a1f-e1fbfd216900 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:24.050108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d72f95-aa99-4fda-8a1f-e1fbfd216900 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:24.051496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 67/134] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:24.052462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d72f95-aa99-4fda-8a1f-e1fbfd216900 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:24.107968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a1c78-5bd7-415a-85ec-c7b8bfc7e3d7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:24.152196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbff78-3d11-4c11-9662-d4ffabe93db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:24.153580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbff78-3d11-4c11-9662-d4ffabe93db9 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:24.154011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbff78-3d11-4c11-9662-d4ffabe93db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:24.154443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbff78-3d11-4c11-9662-d4ffabe93db9 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa/users/2040a306bf664576a535972455e70a9a/roles/23d9f4769fd6424bae83090a691c7f94` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:24.157293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbff78-3d11-4c11-9662-d4ffabe93db9 None admin] RBAC: Authorizing `identity:create_grant(project_id=811b356a854c422f85cddd17b5c478aa, user_id=2040a306bf664576a535972455e70a9a, role_id=23d9f4769fd6424bae83090a691c7f94)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:24.159327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fbff78-3d11-4c11-9662-d4ffabe93db9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:24.178948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 68/135] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:06:23 2022] PUT /identity/v3/projects/811b356a854c422f85cddd17b5c478aa/users/2040a306bf664576a535972455e70a9a/roles/23d9f4769fd6424bae83090a691c7f94 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:24.184103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dd606-5a1c-4cc5-a003-ad63de56ed98 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:24.184103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dd606-5a1c-4cc5-a003-ad63de56ed98 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:24.184103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dd606-5a1c-4cc5-a003-ad63de56ed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:24.228606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5e5dd606-5a1c-4cc5-a003-ad63de56ed98 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:24.232403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e5dd606-5a1c-4cc5-a003-ad63de56ed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:24.271452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d72f95-aa99-4fda-8a1f-e1fbfd216900 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:24.323434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:24.347794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0528e-38a0-42f9-a150-562299175733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:24.348834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0528e-38a0-42f9-a150-562299175733 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:24.349333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0528e-38a0-42f9-a150-562299175733 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:24.349812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0528e-38a0-42f9-a150-562299175733 None admin] PATH_INFO: `/v3/users/2040a306bf664576a535972455e70a9a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:24.353455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0528e-38a0-42f9-a150-562299175733 None admin] RBAC: Authorizing `identity:get_user(user_id=2040a306bf664576a535972455e70a9a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:24.356737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0528e-38a0-42f9-a150-562299175733 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:24.358770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 68/136] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:24 2022] GET /identity/v3/users/2040a306bf664576a535972455e70a9a => generated 275 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:24.364620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0528e-38a0-42f9-a150-562299175733 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:24.381961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 69/137] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:24.434717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:24.514126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0528e-38a0-42f9-a150-562299175733 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:24.525338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dbe86-e748-4120-a599-68c7de9312f0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:24.525814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dbe86-e748-4120-a599-68c7de9312f0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:24.525885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363dbe86-e748-4120-a599-68c7de9312f0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:24.528052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 70/138] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:24.537131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04c055-c1fb-4d0e-886d-1d2c2cc01f5e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:24.537423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04c055-c1fb-4d0e-886d-1d2c2cc01f5e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:24.537701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04c055-c1fb-4d0e-886d-1d2c2cc01f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:24.567969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8e04c055-c1fb-4d0e-886d-1d2c2cc01f5e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:24.569173 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8e04c055-c1fb-4d0e-886d-1d2c2cc01f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:24.572712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9bb4f-5a4c-44c1-84ee-1ffa4a477b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:24.573370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9bb4f-5a4c-44c1-84ee-1ffa4a477b74 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:24.574256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9bb4f-5a4c-44c1-84ee-1ffa4a477b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:24.574479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9bb4f-5a4c-44c1-84ee-1ffa4a477b74 None admin] PATH_INFO: `/v3/projects/8f4741383fee4714834b4003f173590b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:24.575833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9bb4f-5a4c-44c1-84ee-1ffa4a477b74 None admin] RBAC: Authorizing `identity:get_project(project_id=8f4741383fee4714834b4003f173590b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:24.577311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b9bb4f-5a4c-44c1-84ee-1ffa4a477b74 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:24.579138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 69/139] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:24 2022] GET /identity/v3/projects/8f4741383fee4714834b4003f173590b => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:24.589981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9bb4f-5a4c-44c1-84ee-1ffa4a477b74 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:24.724964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 71/140] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:24.731781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04c055-c1fb-4d0e-886d-1d2c2cc01f5e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:24.858060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9bb4f-5a4c-44c1-84ee-1ffa4a477b74 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:24.860056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:24.908313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f73eb-ad1a-49d1-a30d-ad77dd27a11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:24.909379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f73eb-ad1a-49d1-a30d-ad77dd27a11a None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:24.909920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f73eb-ad1a-49d1-a30d-ad77dd27a11a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:24.910394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f73eb-ad1a-49d1-a30d-ad77dd27a11a None admin] PATH_INFO: `/v3/projects/8f4741383fee4714834b4003f173590b/users/2040a306bf664576a535972455e70a9a/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:24.912670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f73eb-ad1a-49d1-a30d-ad77dd27a11a None admin] RBAC: Authorizing `identity:create_grant(project_id=8f4741383fee4714834b4003f173590b, user_id=2040a306bf664576a535972455e70a9a, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:24.914364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f73eb-ad1a-49d1-a30d-ad77dd27a11a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:24.934673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04c055-c1fb-4d0e-886d-1d2c2cc01f5e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:24.938972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 70/141] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:06:24 2022] PUT /identity/v3/projects/8f4741383fee4714834b4003f173590b/users/2040a306bf664576a535972455e70a9a/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:24.956230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:25.004565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012a213-1268-4f4c-a139-eaf2dd42caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:25.005237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012a213-1268-4f4c-a139-eaf2dd42caa5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:25.005517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012a213-1268-4f4c-a139-eaf2dd42caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:25.005794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012a213-1268-4f4c-a139-eaf2dd42caa5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:25.007045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012a213-1268-4f4c-a139-eaf2dd42caa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:25.008578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e012a213-1268-4f4c-a139-eaf2dd42caa5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:25.010414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 72/142] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:25.016485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f73eb-ad1a-49d1-a30d-ad77dd27a11a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:25.094734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012a213-1268-4f4c-a139-eaf2dd42caa5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:25.148757 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f73eb-ad1a-49d1-a30d-ad77dd27a11a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:25.186457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033fb7e-5d60-421b-bdc5-8e428effa202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:25.187489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033fb7e-5d60-421b-bdc5-8e428effa202 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:25.187991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033fb7e-5d60-421b-bdc5-8e428effa202 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:25.188514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033fb7e-5d60-421b-bdc5-8e428effa202 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:25.189377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033fb7e-5d60-421b-bdc5-8e428effa202 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:25.191026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033fb7e-5d60-421b-bdc5-8e428effa202 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:25.224866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 71/143] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:06:25 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:06:25.289345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012a213-1268-4f4c-a139-eaf2dd42caa5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:25.331383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b6533-205f-49e8-b244-6b2d428e7995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:25.332505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b6533-205f-49e8-b244-6b2d428e7995 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:25.333015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b6533-205f-49e8-b244-6b2d428e7995 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:25.333528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b6533-205f-49e8-b244-6b2d428e7995 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:25.347674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b6533-205f-49e8-b244-6b2d428e7995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c51b6533-205f-49e8-b244-6b2d428e7995 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:25.354119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:25.355250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:25.356697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:25.359017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 73/144] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:25.365600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033fb7e-5d60-421b-bdc5-8e428effa202 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:25.388914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:25.487158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033fb7e-5d60-421b-bdc5-8e428effa202 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:25.525504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d567d4-3c12-4e2f-b8e5-7ec0a3979d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:25.527480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d567d4-3c12-4e2f-b8e5-7ec0a3979d84 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:25.528003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d567d4-3c12-4e2f-b8e5-7ec0a3979d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:25.528486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d567d4-3c12-4e2f-b8e5-7ec0a3979d84 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:25.529377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d567d4-3c12-4e2f-b8e5-7ec0a3979d84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:25.529880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d567d4-3c12-4e2f-b8e5-7ec0a3979d84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:25.531866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d567d4-3c12-4e2f-b8e5-7ec0a3979d84 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:25.542710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 72/145] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:25.548890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b6533-205f-49e8-b244-6b2d428e7995 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:25.604088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:25.687443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b6533-205f-49e8-b244-6b2d428e7995 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:25.732437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dc188-26bc-4b7c-81ca-b3588d750144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:25.733192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dc188-26bc-4b7c-81ca-b3588d750144 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:25.733502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dc188-26bc-4b7c-81ca-b3588d750144 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:25.733774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774dc188-26bc-4b7c-81ca-b3588d750144 None admin] PATH_INFO: `/v3/users/2cde2e98da504619a842cc2feb481e74` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:25.740506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dc188-26bc-4b7c-81ca-b3588d750144 None admin] RBAC: Authorizing `identity:get_user(user_id=2cde2e98da504619a842cc2feb481e74)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:25.747512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774dc188-26bc-4b7c-81ca-b3588d750144 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:25.749580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 74/146] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:25 2022] GET /identity/v3/users/2cde2e98da504619a842cc2feb481e74 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:25.755304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d567d4-3c12-4e2f-b8e5-7ec0a3979d84 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:25.879799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:25.902762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d567d4-3c12-4e2f-b8e5-7ec0a3979d84 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:25.948512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512d33a-4f6f-47c7-86bd-2249821329c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:25.949169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512d33a-4f6f-47c7-86bd-2249821329c8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:25.949451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512d33a-4f6f-47c7-86bd-2249821329c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:25.949726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512d33a-4f6f-47c7-86bd-2249821329c8 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:25.950887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512d33a-4f6f-47c7-86bd-2249821329c8 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:25.952420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3512d33a-4f6f-47c7-86bd-2249821329c8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:25.954057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 73/147] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:25 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:25.959947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dc188-26bc-4b7c-81ca-b3588d750144 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:26.090873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774dc188-26bc-4b7c-81ca-b3588d750144 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:26.100673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:26.129633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98deff5-55a8-49f4-ac49-0b1acf3560c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:26.130292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98deff5-55a8-49f4-ac49-0b1acf3560c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:26.130567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98deff5-55a8-49f4-ac49-0b1acf3560c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:26.130836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98deff5-55a8-49f4-ac49-0b1acf3560c9 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa/users/2cde2e98da504619a842cc2feb481e74/roles/9dbcf5e8a10a4728888b9f4e923f5814` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:26.134688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98deff5-55a8-49f4-ac49-0b1acf3560c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=811b356a854c422f85cddd17b5c478aa, user_id=2cde2e98da504619a842cc2feb481e74, role_id=9dbcf5e8a10a4728888b9f4e923f5814)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:26.136709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98deff5-55a8-49f4-ac49-0b1acf3560c9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:26.154128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 75/148] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:06:25 2022] PUT /identity/v3/projects/811b356a854c422f85cddd17b5c478aa/users/2cde2e98da504619a842cc2feb481e74/roles/9dbcf5e8a10a4728888b9f4e923f5814 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:26.758457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aea8f-b3f5-4ab2-afcc-1ff79a516a6a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:26.758984 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aea8f-b3f5-4ab2-afcc-1ff79a516a6a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:26.758984 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7aea8f-b3f5-4ab2-afcc-1ff79a516a6a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:26.760518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 74/149] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:26.767221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a384a0-2aeb-4da2-bd9b-5f8e85daeac2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:26.767599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a384a0-2aeb-4da2-bd9b-5f8e85daeac2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:26.767812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a384a0-2aeb-4da2-bd9b-5f8e85daeac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:26.797137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-09a384a0-2aeb-4da2-bd9b-5f8e85daeac2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:26.798491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-09a384a0-2aeb-4da2-bd9b-5f8e85daeac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:26.878089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 76/150] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:27.283490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7a413-5c26-485b-804e-3b25189095e3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:27.283899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7a413-5c26-485b-804e-3b25189095e3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:27.284073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7a413-5c26-485b-804e-3b25189095e3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:27.285629 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 75/151] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:27.293269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99a6aa-2f76-47c7-bda7-84fb9992b328 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:27.293545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99a6aa-2f76-47c7-bda7-84fb9992b328 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:27.293817 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99a6aa-2f76-47c7-bda7-84fb9992b328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:27.316760 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cb99a6aa-2f76-47c7-bda7-84fb9992b328 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:27.317958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cb99a6aa-2f76-47c7-bda7-84fb9992b328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:27.409371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 77/152] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:27.464099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7a413-5c26-485b-804e-3b25189095e3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:27.562057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c6547-bbe2-4f01-9840-18c0ea976039 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:27.562344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c6547-bbe2-4f01-9840-18c0ea976039 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:27.562498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c6547-bbe2-4f01-9840-18c0ea976039 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:27.564583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 78/153] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:27.570688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948efa8-3e1d-4e48-9fab-ede37bba76eb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:27.570965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948efa8-3e1d-4e48-9fab-ede37bba76eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:27.571236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948efa8-3e1d-4e48-9fab-ede37bba76eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:27.612951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0948efa8-3e1d-4e48-9fab-ede37bba76eb None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:27.614175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0948efa8-3e1d-4e48-9fab-ede37bba76eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:27.617480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7a413-5c26-485b-804e-3b25189095e3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:27.659788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:27.688522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a017431-0076-4f49-b457-8e05e6297697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:27.689193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a017431-0076-4f49-b457-8e05e6297697 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:27.689478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a017431-0076-4f49-b457-8e05e6297697 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:27.689750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a017431-0076-4f49-b457-8e05e6297697 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:27.705608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a017431-0076-4f49-b457-8e05e6297697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-7a017431-0076-4f49-b457-8e05e6297697 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:27.711024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:27.713181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:27.716004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 76/154] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:27.723878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a017431-0076-4f49-b457-8e05e6297697 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:27.737162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:27.785417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 79/155] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:27.873712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a017431-0076-4f49-b457-8e05e6297697 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:27.923251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fe823-9661-46d4-b69e-4eca58dcae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:27.923899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fe823-9661-46d4-b69e-4eca58dcae56 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:27.924186 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fe823-9661-46d4-b69e-4eca58dcae56 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:27.924462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92fe823-9661-46d4-b69e-4eca58dcae56 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:27.925133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fe823-9661-46d4-b69e-4eca58dcae56 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:27.925436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fe823-9661-46d4-b69e-4eca58dcae56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:27.926828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92fe823-9661-46d4-b69e-4eca58dcae56 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:27.936463 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 77/156] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:27.942092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948efa8-3e1d-4e48-9fab-ede37bba76eb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:27.955647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fe823-9661-46d4-b69e-4eca58dcae56 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:28.053227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:28.096512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92fe823-9661-46d4-b69e-4eca58dcae56 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:28.150854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0948efa8-3e1d-4e48-9fab-ede37bba76eb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:28.154146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113610f-3e8d-4ac1-a8bc-81a510c95916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:28.154807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113610f-3e8d-4ac1-a8bc-81a510c95916 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:28.155093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113610f-3e8d-4ac1-a8bc-81a510c95916 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:28.155364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113610f-3e8d-4ac1-a8bc-81a510c95916 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:28.176414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113610f-3e8d-4ac1-a8bc-81a510c95916 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3113610f-3e8d-4ac1-a8bc-81a510c95916 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:28.181978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:28.183089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:28.183969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 78/157] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:06:28.191656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113610f-3e8d-4ac1-a8bc-81a510c95916 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:28.222351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de7423-9830-4844-a429-8b78b5cc787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:28.223384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de7423-9830-4844-a429-8b78b5cc787f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:28.224004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de7423-9830-4844-a429-8b78b5cc787f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:28.224543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de7423-9830-4844-a429-8b78b5cc787f None admin] PATH_INFO: `/v3/users/2040a306bf664576a535972455e70a9a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:28.227405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de7423-9830-4844-a429-8b78b5cc787f None admin] RBAC: Authorizing `identity:get_user(user_id=2040a306bf664576a535972455e70a9a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:28.229199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de7423-9830-4844-a429-8b78b5cc787f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:28.231086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 80/158] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:27 2022] GET /identity/v3/users/2040a306bf664576a535972455e70a9a => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:28.266327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de7423-9830-4844-a429-8b78b5cc787f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:28.329496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:28.340116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113610f-3e8d-4ac1-a8bc-81a510c95916 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:28.389889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:28.398383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5cf92-642d-4c1d-afd4-57fe3dd3275c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:28.399012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5cf92-642d-4c1d-afd4-57fe3dd3275c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:28.399296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5cf92-642d-4c1d-afd4-57fe3dd3275c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:28.399772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5cf92-642d-4c1d-afd4-57fe3dd3275c None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:28.400366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5cf92-642d-4c1d-afd4-57fe3dd3275c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:28.400659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5cf92-642d-4c1d-afd4-57fe3dd3275c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:28.402639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5cf92-642d-4c1d-afd4-57fe3dd3275c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:28.416743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 79/159] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:28.424029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5cf92-642d-4c1d-afd4-57fe3dd3275c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:28.478102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de7423-9830-4844-a429-8b78b5cc787f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:28.552784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a2da4-55a9-4c61-9bb1-3746d49a1337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:28.553690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a2da4-55a9-4c61-9bb1-3746d49a1337 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:28.553980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a2da4-55a9-4c61-9bb1-3746d49a1337 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:28.554256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a2da4-55a9-4c61-9bb1-3746d49a1337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:28.670842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5cf92-642d-4c1d-afd4-57fe3dd3275c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:28.685049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a2da4-55a9-4c61-9bb1-3746d49a1337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:28.686559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a2da4-55a9-4c61-9bb1-3746d49a1337 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:28.721462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881fcf9-616a-4c6a-a400-f8d5079601f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:28.722346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881fcf9-616a-4c6a-a400-f8d5079601f2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:28.722627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881fcf9-616a-4c6a-a400-f8d5079601f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:28.722917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881fcf9-616a-4c6a-a400-f8d5079601f2 None admin] PATH_INFO: `/v3/users/661f5d19be0f436ca14e6e9b440ca709` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:28.727147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881fcf9-616a-4c6a-a400-f8d5079601f2 None admin] RBAC: Authorizing `identity:get_user(user_id=661f5d19be0f436ca14e6e9b440ca709)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:28.728704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881fcf9-616a-4c6a-a400-f8d5079601f2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:28.732134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 80/160] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:28 2022] GET /identity/v3/users/661f5d19be0f436ca14e6e9b440ca709 => generated 247 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:28.733197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881fcf9-616a-4c6a-a400-f8d5079601f2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:28.739506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:28.847140 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:28.910670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881fcf9-616a-4c6a-a400-f8d5079601f2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:28.926950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 81/161] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 12:06:28.928563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a2da4-55a9-4c61-9bb1-3746d49a1337 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:28.959190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3b3c-ee39-43b7-8277-bee4d1c08e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:28.961378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3b3c-ee39-43b7-8277-bee4d1c08e69 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:28.963876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3b3c-ee39-43b7-8277-bee4d1c08e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:28.964378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd3b3c-ee39-43b7-8277-bee4d1c08e69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:28.994558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd3b3c-ee39-43b7-8277-bee4d1c08e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e1dd3b3c-ee39-43b7-8277-bee4d1c08e69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:29.004990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:29.005955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:29.006715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:29.014632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 81/162] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:06:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:29.015776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3b3c-ee39-43b7-8277-bee4d1c08e69 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:29.127150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a2da4-55a9-4c61-9bb1-3746d49a1337 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:29.177951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:29.198815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f90c98-8eb4-486f-b7d1-ec37eac4259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:29.199821 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f90c98-8eb4-486f-b7d1-ec37eac4259e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:29.200294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f90c98-8eb4-486f-b7d1-ec37eac4259e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:29.200774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f90c98-8eb4-486f-b7d1-ec37eac4259e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:29.289911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd3b3c-ee39-43b7-8277-bee4d1c08e69 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:29.317272 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f90c98-8eb4-486f-b7d1-ec37eac4259e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:29.319058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f90c98-8eb4-486f-b7d1-ec37eac4259e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:29.336409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:29.376506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1baced-66c4-481b-98ee-18406e0f3102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:29.377530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1baced-66c4-481b-98ee-18406e0f3102 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:29.378036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1baced-66c4-481b-98ee-18406e0f3102 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:29.378691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1baced-66c4-481b-98ee-18406e0f3102 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:29.382434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1baced-66c4-481b-98ee-18406e0f3102 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:29.384197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1baced-66c4-481b-98ee-18406e0f3102 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:29.386182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 82/163] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:29 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:29.387549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24068b-ebb6-493c-ad26-19182125a884 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:29.388060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24068b-ebb6-493c-ad26-19182125a884 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:29.388534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24068b-ebb6-493c-ad26-19182125a884 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:29.389942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 83/164] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:29.390855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24068b-ebb6-493c-ad26-19182125a884 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:29.491693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 82/165] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:29.492898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667bb92-62f8-4059-84d1-09d31b0b0411 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:29.493185 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667bb92-62f8-4059-84d1-09d31b0b0411 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:29.493450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667bb92-62f8-4059-84d1-09d31b0b0411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:29.528864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7667bb92-62f8-4059-84d1-09d31b0b0411 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:29.530475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7667bb92-62f8-4059-84d1-09d31b0b0411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:29.552445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24068b-ebb6-493c-ad26-19182125a884 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:29.565594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:29.609897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557322a6-6876-4acf-bb0b-7ef3df8034b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:29.610521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557322a6-6876-4acf-bb0b-7ef3df8034b9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:29.610787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557322a6-6876-4acf-bb0b-7ef3df8034b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:29.611047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557322a6-6876-4acf-bb0b-7ef3df8034b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:29.611721 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557322a6-6876-4acf-bb0b-7ef3df8034b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:29.612043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557322a6-6876-4acf-bb0b-7ef3df8034b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:29.613455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557322a6-6876-4acf-bb0b-7ef3df8034b9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:29.632142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 84/166] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:06:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:29.633365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557322a6-6876-4acf-bb0b-7ef3df8034b9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:29.653361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 83/167] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:29.656122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667bb92-62f8-4059-84d1-09d31b0b0411 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:29.734929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:29.830120 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667bb92-62f8-4059-84d1-09d31b0b0411 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:29.872066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557322a6-6876-4acf-bb0b-7ef3df8034b9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:29.892071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7478c9-0913-4083-a4bb-b4f589b8cccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:29.893015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7478c9-0913-4083-a4bb-b4f589b8cccd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:29.893445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7478c9-0913-4083-a4bb-b4f589b8cccd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:29.893866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7478c9-0913-4083-a4bb-b4f589b8cccd None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:29.895306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7478c9-0913-4083-a4bb-b4f589b8cccd None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:29.896970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7478c9-0913-4083-a4bb-b4f589b8cccd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:29.899180 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 84/168] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:29 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:29.900240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7478c9-0913-4083-a4bb-b4f589b8cccd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:29.928510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467d7a0-e09b-4818-8aa5-b03c4a025767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:29.929145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467d7a0-e09b-4818-8aa5-b03c4a025767 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:29.929428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467d7a0-e09b-4818-8aa5-b03c4a025767 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:29.929704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467d7a0-e09b-4818-8aa5-b03c4a025767 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:29.930447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467d7a0-e09b-4818-8aa5-b03c4a025767 None admin] RBAC: Adding query filter params (role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=811b356a854c422f85cddd17b5c478aa, user.id=2040a306bf664576a535972455e70a9a) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:29.930758 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467d7a0-e09b-4818-8aa5-b03c4a025767 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:29.932270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467d7a0-e09b-4818-8aa5-b03c4a025767 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:29.943827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 85/169] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:29 2022] GET /identity/v3/role_assignments?user.id=2040a306bf664576a535972455e70a9a&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 608 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:29.943827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467d7a0-e09b-4818-8aa5-b03c4a025767 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:29.953748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:30.083712 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7478c9-0913-4083-a4bb-b4f589b8cccd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:30.109687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467d7a0-e09b-4818-8aa5-b03c4a025767 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:30.129383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:30.144786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c4ab6-4f0d-47ce-8afa-3b531a231d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:30.145965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c4ab6-4f0d-47ce-8afa-3b531a231d75 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.146564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c4ab6-4f0d-47ce-8afa-3b531a231d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.147054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c4ab6-4f0d-47ce-8afa-3b531a231d75 None admin] PATH_INFO: `/v3/users/2040a306bf664576a535972455e70a9a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.148582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c4ab6-4f0d-47ce-8afa-3b531a231d75 None admin] RBAC: Authorizing `identity:get_user(user_id=2040a306bf664576a535972455e70a9a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:30.150449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c4ab6-4f0d-47ce-8afa-3b531a231d75 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:30.153301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 85/170] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:29 2022] GET /identity/v3/users/2040a306bf664576a535972455e70a9a => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:30.158812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c4ab6-4f0d-47ce-8afa-3b531a231d75 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:30.159359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3b741-32b4-4321-a3aa-7586e41e0818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:30.160012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3b741-32b4-4321-a3aa-7586e41e0818 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.160263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3b741-32b4-4321-a3aa-7586e41e0818 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.160534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3b741-32b4-4321-a3aa-7586e41e0818 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.161262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3b741-32b4-4321-a3aa-7586e41e0818 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=811b356a854c422f85cddd17b5c478aa, user.id=661f5d19be0f436ca14e6e9b440ca709) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:30.161566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3b741-32b4-4321-a3aa-7586e41e0818 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:30.163036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3b741-32b4-4321-a3aa-7586e41e0818 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:30.170117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 86/171] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:29 2022] GET /identity/v3/role_assignments?user.id=661f5d19be0f436ca14e6e9b440ca709&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:30.226269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65221013-c4d0-464b-920a-9981ea605c58 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.226585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65221013-c4d0-464b-920a-9981ea605c58 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.226826 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65221013-c4d0-464b-920a-9981ea605c58 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.228502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 87/172] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:30.234321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095e90d-3a83-4778-aa9f-e57f44c3ff6e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.234644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095e90d-3a83-4778-aa9f-e57f44c3ff6e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.234871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095e90d-3a83-4778-aa9f-e57f44c3ff6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.266009 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4095e90d-3a83-4778-aa9f-e57f44c3ff6e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:30.267260 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4095e90d-3a83-4778-aa9f-e57f44c3ff6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:30.323546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:30.341323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c4ab6-4f0d-47ce-8afa-3b531a231d75 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:30.377178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 88/173] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:30.378066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095e90d-3a83-4778-aa9f-e57f44c3ff6e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:30.417836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7677bf-37b3-4ba2-aba3-4596bc661208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:30.418472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7677bf-37b3-4ba2-aba3-4596bc661208 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.418752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7677bf-37b3-4ba2-aba3-4596bc661208 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.419027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7677bf-37b3-4ba2-aba3-4596bc661208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.599027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095e90d-3a83-4778-aa9f-e57f44c3ff6e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:30.610199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:30.620623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7677bf-37b3-4ba2-aba3-4596bc661208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:30.622149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7677bf-37b3-4ba2-aba3-4596bc661208 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:30.641069 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b0fe6-a270-4b07-99f5-3adeb709ca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:30.641789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0fe6-a270-4b07-99f5-3adeb709ca7e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.642072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0fe6-a270-4b07-99f5-3adeb709ca7e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.642348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b0fe6-a270-4b07-99f5-3adeb709ca7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.655210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b0fe6-a270-4b07-99f5-3adeb709ca7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d62b0fe6-a270-4b07-99f5-3adeb709ca7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:30.661025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:30.661905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:30.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 89/174] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:30.665845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcf444-cd7f-45f2-baed-d343caca017a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.666128 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcf444-cd7f-45f2-baed-d343caca017a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.666402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcf444-cd7f-45f2-baed-d343caca017a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.668686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 90/175] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:30.669698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcf444-cd7f-45f2-baed-d343caca017a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:30.760689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:30.774897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 86/176] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 12:06:30.777516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644c710-a28d-4dc3-8502-3bc24299274b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.778018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644c710-a28d-4dc3-8502-3bc24299274b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.778487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644c710-a28d-4dc3-8502-3bc24299274b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.798315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1644c710-a28d-4dc3-8502-3bc24299274b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:30.799983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1644c710-a28d-4dc3-8502-3bc24299274b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:30.838972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcf444-cd7f-45f2-baed-d343caca017a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:30.892896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089b954-1d04-4de0-bd43-7d369d8ed3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:30.893434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089b954-1d04-4de0-bd43-7d369d8ed3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.893715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089b954-1d04-4de0-bd43-7d369d8ed3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.893976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089b954-1d04-4de0-bd43-7d369d8ed3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.894720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089b954-1d04-4de0-bd43-7d369d8ed3c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:30.895023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089b954-1d04-4de0-bd43-7d369d8ed3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:30.897010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089b954-1d04-4de0-bd43-7d369d8ed3c1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:30.901445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 87/177] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:30.907342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 91/178] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:30.907963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089b954-1d04-4de0-bd43-7d369d8ed3c1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:30.909369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749db4f-6ef8-4be8-9217-83e733304d1b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:30.909862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749db4f-6ef8-4be8-9217-83e733304d1b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:30.910327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749db4f-6ef8-4be8-9217-83e733304d1b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:30.912336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 88/179] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:30.913373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749db4f-6ef8-4be8-9217-83e733304d1b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:30.972331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:31.041494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749db4f-6ef8-4be8-9217-83e733304d1b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:31.043158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089b954-1d04-4de0-bd43-7d369d8ed3c1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:31.068128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:31.084772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c12b2-5a03-4243-ac7b-a6dc1c6b7e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:31.085226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc9b3e-adb3-4ec5-84c8-f77c815fc2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:31.085873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc9b3e-adb3-4ec5-84c8-f77c815fc2de None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.086156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc9b3e-adb3-4ec5-84c8-f77c815fc2de None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.086283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c12b2-5a03-4243-ac7b-a6dc1c6b7e6a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.086438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc9b3e-adb3-4ec5-84c8-f77c815fc2de None admin] PATH_INFO: `/v3/users/2040a306bf664576a535972455e70a9a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.086894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c12b2-5a03-4243-ac7b-a6dc1c6b7e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.087372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c12b2-5a03-4243-ac7b-a6dc1c6b7e6a None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.087624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc9b3e-adb3-4ec5-84c8-f77c815fc2de None admin] RBAC: Authorizing `identity:get_user(user_id=2040a306bf664576a535972455e70a9a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:31.089488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c12b2-5a03-4243-ac7b-a6dc1c6b7e6a None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:31.089770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc9b3e-adb3-4ec5-84c8-f77c815fc2de None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:31.091792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 89/180] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:30 2022] GET /identity/v3/users/2040a306bf664576a535972455e70a9a => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:31.092187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c12b2-5a03-4243-ac7b-a6dc1c6b7e6a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:31.092861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ceebdf-4fc4-4c0c-bd9c-e73556c7f793 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.093172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ceebdf-4fc4-4c0c-bd9c-e73556c7f793 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.093481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ceebdf-4fc4-4c0c-bd9c-e73556c7f793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.094716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 92/181] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:30 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:31.095715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c12b2-5a03-4243-ac7b-a6dc1c6b7e6a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:31.122582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-76ceebdf-4fc4-4c0c-bd9c-e73556c7f793 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:31.124120 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-76ceebdf-4fc4-4c0c-bd9c-e73556c7f793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:31.200485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:31.206061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c12b2-5a03-4243-ac7b-a6dc1c6b7e6a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:31.223893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 90/182] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:31.224780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ceebdf-4fc4-4c0c-bd9c-e73556c7f793 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:31.259282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102de91-5c20-42e1-9ba0-cb769afd2e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:31.260482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102de91-5c20-42e1-9ba0-cb769afd2e0f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.261374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102de91-5c20-42e1-9ba0-cb769afd2e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.261738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102de91-5c20-42e1-9ba0-cb769afd2e0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.276268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102de91-5c20-42e1-9ba0-cb769afd2e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2102de91-5c20-42e1-9ba0-cb769afd2e0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:31.283424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:31.284309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:31.285095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 93/183] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:06:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:31.285737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102de91-5c20-42e1-9ba0-cb769afd2e0f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:31.345236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ceebdf-4fc4-4c0c-bd9c-e73556c7f793 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:31.369804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:31.384181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b5839-39ed-4003-bffa-e6ee76091c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:31.384794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b5839-39ed-4003-bffa-e6ee76091c6e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.385065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b5839-39ed-4003-bffa-e6ee76091c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.385335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b5839-39ed-4003-bffa-e6ee76091c6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.412795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102de91-5c20-42e1-9ba0-cb769afd2e0f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:31.449263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a42f-1d2c-4292-8a1e-394b6f8f6ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:31.449880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a42f-1d2c-4292-8a1e-394b6f8f6ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.450159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a42f-1d2c-4292-8a1e-394b6f8f6ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.450459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67a42f-1d2c-4292-8a1e-394b6f8f6ab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.451296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67a42f-1d2c-4292-8a1e-394b6f8f6ab1 None admin] RBAC: Adding query filter params (role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=811b356a854c422f85cddd17b5c478aa, user.id=2040a306bf664576a535972455e70a9a) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:31.451548 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b5839-39ed-4003-bffa-e6ee76091c6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:31.451652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67a42f-1d2c-4292-8a1e-394b6f8f6ab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:31.453045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67a42f-1d2c-4292-8a1e-394b6f8f6ab1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:31.453149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b5839-39ed-4003-bffa-e6ee76091c6e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:31.463788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 94/184] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:31 2022] GET /identity/v3/role_assignments?user.id=2040a306bf664576a535972455e70a9a&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:31.463788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a42f-1d2c-4292-8a1e-394b6f8f6ab1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:31.499540 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:31.602612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67a42f-1d2c-4292-8a1e-394b6f8f6ab1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:31.610199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 91/185] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:31.611328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b5839-39ed-4003-bffa-e6ee76091c6e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:31.637878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:31.648419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a359a79-edb1-46d3-9c66-a6f4c8ef70c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:31.649060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a359a79-edb1-46d3-9c66-a6f4c8ef70c1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.649337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a359a79-edb1-46d3-9c66-a6f4c8ef70c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.649610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a359a79-edb1-46d3-9c66-a6f4c8ef70c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.661102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a359a79-edb1-46d3-9c66-a6f4c8ef70c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-8a359a79-edb1-46d3-9c66-a6f4c8ef70c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:31.667905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:31.668785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:31.669831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 95/186] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:31.670621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ecbc-bd58-43ad-877b-6678bad48a05 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.670621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ecbc-bd58-43ad-877b-6678bad48a05 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.670621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ecbc-bd58-43ad-877b-6678bad48a05 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.671693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 96/187] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:31.676694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762ecbc-bd58-43ad-877b-6678bad48a05 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:31.740145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b5839-39ed-4003-bffa-e6ee76091c6e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:31.772778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:31.777973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453af68-8f88-42cd-9177-42a6b8dfdc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:31.778582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453af68-8f88-42cd-9177-42a6b8dfdc8b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.778862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453af68-8f88-42cd-9177-42a6b8dfdc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.779140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453af68-8f88-42cd-9177-42a6b8dfdc8b None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.779833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453af68-8f88-42cd-9177-42a6b8dfdc8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:31.780140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453af68-8f88-42cd-9177-42a6b8dfdc8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:31.781514 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9453af68-8f88-42cd-9177-42a6b8dfdc8b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:31.789739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 92/188] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:31.790534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453af68-8f88-42cd-9177-42a6b8dfdc8b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:31.791865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762ecbc-bd58-43ad-877b-6678bad48a05 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:31.825596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa561a-f77f-4b91-9fe0-c69daa4fa1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:31.826222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa561a-f77f-4b91-9fe0-c69daa4fa1fe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.826498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa561a-f77f-4b91-9fe0-c69daa4fa1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.826767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa561a-f77f-4b91-9fe0-c69daa4fa1fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.838182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa561a-f77f-4b91-9fe0-c69daa4fa1fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-12aa561a-f77f-4b91-9fe0-c69daa4fa1fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:31.843325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:31.844243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:31.845042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 97/189] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:06:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:31.845906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb06c7f-746e-43d9-9c51-c6e9ff868529 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.845906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb06c7f-746e-43d9-9c51-c6e9ff868529 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.846182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb06c7f-746e-43d9-9c51-c6e9ff868529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.863801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4bb06c7f-746e-43d9-9c51-c6e9ff868529 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:31.864994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4bb06c7f-746e-43d9-9c51-c6e9ff868529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:31.885076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:31.899070 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453af68-8f88-42cd-9177-42a6b8dfdc8b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:31.936794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192efd96-57cc-450a-9ad2-d71bc10980ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:31.937937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192efd96-57cc-450a-9ad2-d71bc10980ce None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:31.937937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192efd96-57cc-450a-9ad2-d71bc10980ce None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:31.937937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192efd96-57cc-450a-9ad2-d71bc10980ce None admin] PATH_INFO: `/v3/projects/8f4741383fee4714834b4003f173590b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:31.939038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192efd96-57cc-450a-9ad2-d71bc10980ce None admin] RBAC: Authorizing `identity:get_project(project_id=8f4741383fee4714834b4003f173590b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:31.940522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192efd96-57cc-450a-9ad2-d71bc10980ce None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:31.942149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 93/190] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:31 2022] GET /identity/v3/projects/8f4741383fee4714834b4003f173590b => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:31.944566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192efd96-57cc-450a-9ad2-d71bc10980ce None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:31.956741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 98/191] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:31.957408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb06c7f-746e-43d9-9c51-c6e9ff868529 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.041515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:32.056308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192efd96-57cc-450a-9ad2-d71bc10980ce None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.078810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb06c7f-746e-43d9-9c51-c6e9ff868529 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.097245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086eade-553d-4953-bdd2-33f24cd6c8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.097861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086eade-553d-4953-bdd2-33f24cd6c8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.098140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086eade-553d-4953-bdd2-33f24cd6c8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.098414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086eade-553d-4953-bdd2-33f24cd6c8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.099082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086eade-553d-4953-bdd2-33f24cd6c8a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:32.099394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086eade-553d-4953-bdd2-33f24cd6c8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.100840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086eade-553d-4953-bdd2-33f24cd6c8a2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.109750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 94/192] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:32.110634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086eade-553d-4953-bdd2-33f24cd6c8a2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.114704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961506b-2fc0-4e72-9b90-eb4ef7d1503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.115341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961506b-2fc0-4e72-9b90-eb4ef7d1503c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.115623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961506b-2fc0-4e72-9b90-eb4ef7d1503c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.115934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961506b-2fc0-4e72-9b90-eb4ef7d1503c None admin] PATH_INFO: `/v3/users/864a584c698748e89d6245905b6f26fb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.128881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:32.134735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961506b-2fc0-4e72-9b90-eb4ef7d1503c None admin] RBAC: Authorizing `identity:get_user(user_id=864a584c698748e89d6245905b6f26fb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.136281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961506b-2fc0-4e72-9b90-eb4ef7d1503c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.138109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 99/193] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:31 2022] GET /identity/v3/users/864a584c698748e89d6245905b6f26fb => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:32.138706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961506b-2fc0-4e72-9b90-eb4ef7d1503c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.224791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086eade-553d-4953-bdd2-33f24cd6c8a2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.251695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961506b-2fc0-4e72-9b90-eb4ef7d1503c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.261624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9e395-94c0-4fc0-bab4-3a72f4069fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.262230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9e395-94c0-4fc0-bab4-3a72f4069fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.262509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9e395-94c0-4fc0-bab4-3a72f4069fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.262779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9e395-94c0-4fc0-bab4-3a72f4069fe1 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.263441 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9e395-94c0-4fc0-bab4-3a72f4069fe1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:32.263749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9e395-94c0-4fc0-bab4-3a72f4069fe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.265184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9e395-94c0-4fc0-bab4-3a72f4069fe1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.273146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 95/194] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:06:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:32.273946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9e395-94c0-4fc0-bab4-3a72f4069fe1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.285880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:32.292133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f4e2-01ea-4bc1-9818-59ee05409924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.292690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f4e2-01ea-4bc1-9818-59ee05409924 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.292967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f4e2-01ea-4bc1-9818-59ee05409924 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.293240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682f4e2-01ea-4bc1-9818-59ee05409924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.293963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682f4e2-01ea-4bc1-9818-59ee05409924 None admin] RBAC: Adding query filter params (role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=8f4741383fee4714834b4003f173590b, user.id=2040a306bf664576a535972455e70a9a) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:32.294281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682f4e2-01ea-4bc1-9818-59ee05409924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.295709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682f4e2-01ea-4bc1-9818-59ee05409924 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.302044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 100/195] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:32 2022] GET /identity/v3/role_assignments?user.id=2040a306bf664576a535972455e70a9a&scope.project.id=8f4741383fee4714834b4003f173590b&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:32.303002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f4e2-01ea-4bc1-9818-59ee05409924 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.361082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:32.392446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9e395-94c0-4fc0-bab4-3a72f4069fe1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.423634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682f4e2-01ea-4bc1-9818-59ee05409924 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.430720 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb17118-66cc-4c48-89f6-f185c3e15892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.431973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb17118-66cc-4c48-89f6-f185c3e15892 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.432268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb17118-66cc-4c48-89f6-f185c3e15892 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.432551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb17118-66cc-4c48-89f6-f185c3e15892 None admin] PATH_INFO: `/v3/users/2cde2e98da504619a842cc2feb481e74` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.434736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb17118-66cc-4c48-89f6-f185c3e15892 None admin] RBAC: Authorizing `identity:get_user(user_id=2cde2e98da504619a842cc2feb481e74)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.436649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb17118-66cc-4c48-89f6-f185c3e15892 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.438549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 96/196] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:32 2022] GET /identity/v3/users/2cde2e98da504619a842cc2feb481e74 => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:32.439226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb17118-66cc-4c48-89f6-f185c3e15892 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.468895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090c658-7f1d-4d16-9c3a-15e0cbc7ab3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.469883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090c658-7f1d-4d16-9c3a-15e0cbc7ab3d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.470165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090c658-7f1d-4d16-9c3a-15e0cbc7ab3d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.470587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090c658-7f1d-4d16-9c3a-15e0cbc7ab3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.546390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:32.556735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090c658-7f1d-4d16-9c3a-15e0cbc7ab3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.559803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090c658-7f1d-4d16-9c3a-15e0cbc7ab3d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.561966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb17118-66cc-4c48-89f6-f185c3e15892 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.596896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8fa77-ffdf-4aaa-a37d-866e4815966a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.597531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8fa77-ffdf-4aaa-a37d-866e4815966a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.597810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8fa77-ffdf-4aaa-a37d-866e4815966a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.598086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8fa77-ffdf-4aaa-a37d-866e4815966a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.599199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8fa77-ffdf-4aaa-a37d-866e4815966a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.600694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8fa77-ffdf-4aaa-a37d-866e4815966a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.602275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 97/197] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:32.603042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8fa77-ffdf-4aaa-a37d-866e4815966a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.616369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:32.659615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 101/198] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:32.659615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090c658-7f1d-4d16-9c3a-15e0cbc7ab3d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.706321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8fa77-ffdf-4aaa-a37d-866e4815966a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.740622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8a7e4-ae3b-49a2-b2a8-e05dc2b05b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.741765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8a7e4-ae3b-49a2-b2a8-e05dc2b05b72 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.742043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8a7e4-ae3b-49a2-b2a8-e05dc2b05b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.742332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8a7e4-ae3b-49a2-b2a8-e05dc2b05b72 None admin] PATH_INFO: `/v3/users/864a584c698748e89d6245905b6f26fb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.743449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8a7e4-ae3b-49a2-b2a8-e05dc2b05b72 None admin] RBAC: Authorizing `identity:get_user(user_id=864a584c698748e89d6245905b6f26fb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.747043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8a7e4-ae3b-49a2-b2a8-e05dc2b05b72 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.748683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 98/199] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:32 2022] GET /identity/v3/users/864a584c698748e89d6245905b6f26fb => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:32.749426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8a7e4-ae3b-49a2-b2a8-e05dc2b05b72 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.753035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:32.762976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090c658-7f1d-4d16-9c3a-15e0cbc7ab3d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.796362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cc36f-58a5-4721-92b7-682fc69cfd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.797029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cc36f-58a5-4721-92b7-682fc69cfd80 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.797247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cc36f-58a5-4721-92b7-682fc69cfd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.797516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cc36f-58a5-4721-92b7-682fc69cfd80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.832386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:32.854857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8a7e4-ae3b-49a2-b2a8-e05dc2b05b72 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:32.863909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cc36f-58a5-4721-92b7-682fc69cfd80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.865379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cc36f-58a5-4721-92b7-682fc69cfd80 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.889056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131ef45-bfd0-4c55-838e-146cfe310f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:32.889667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131ef45-bfd0-4c55-838e-146cfe310f44 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:32.889945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131ef45-bfd0-4c55-838e-146cfe310f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:32.890214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131ef45-bfd0-4c55-838e-146cfe310f44 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:32.890873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131ef45-bfd0-4c55-838e-146cfe310f44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:32.892416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7131ef45-bfd0-4c55-838e-146cfe310f44 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:32.918840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 99/200] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:06:32 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:32.919661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131ef45-bfd0-4c55-838e-146cfe310f44 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:32.964685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:32.978458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 102/201] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:32.978458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cc36f-58a5-4721-92b7-682fc69cfd80 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:33.047127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131ef45-bfd0-4c55-838e-146cfe310f44 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:33.091767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396898e-7cf5-4a5d-90ff-12501b413d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:33.092469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396898e-7cf5-4a5d-90ff-12501b413d88 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:33.092724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396898e-7cf5-4a5d-90ff-12501b413d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:33.092978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396898e-7cf5-4a5d-90ff-12501b413d88 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:33.095132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396898e-7cf5-4a5d-90ff-12501b413d88 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:33.095288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cc36f-58a5-4721-92b7-682fc69cfd80 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:33.097007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396898e-7cf5-4a5d-90ff-12501b413d88 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:33.098561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 100/202] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:32 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:33.098967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396898e-7cf5-4a5d-90ff-12501b413d88 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:33.101987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:33.140269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8ec2b-934e-434e-8ec2-285ff7fbe007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:33.141257 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8ec2b-934e-434e-8ec2-285ff7fbe007 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:33.141715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8ec2b-934e-434e-8ec2-285ff7fbe007 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:33.142164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8ec2b-934e-434e-8ec2-285ff7fbe007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:33.227070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:33.237338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8ec2b-934e-434e-8ec2-285ff7fbe007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:33.237940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396898e-7cf5-4a5d-90ff-12501b413d88 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:33.239432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8ec2b-934e-434e-8ec2-285ff7fbe007 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:33.285201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b637e9-3ddb-4daf-a62a-4d75dd8293be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:33.285847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b637e9-3ddb-4daf-a62a-4d75dd8293be None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:33.286126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b637e9-3ddb-4daf-a62a-4d75dd8293be None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:33.286395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b637e9-3ddb-4daf-a62a-4d75dd8293be None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:33.288091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b637e9-3ddb-4daf-a62a-4d75dd8293be None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:33.289928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b637e9-3ddb-4daf-a62a-4d75dd8293be None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:33.292899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 101/203] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:33 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:33.294168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b637e9-3ddb-4daf-a62a-4d75dd8293be None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:33.360152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 103/204] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:33.365538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8ec2b-934e-434e-8ec2-285ff7fbe007 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:33.393934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:33.439460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b637e9-3ddb-4daf-a62a-4d75dd8293be None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:33.470890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:33.478790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8ec2b-934e-434e-8ec2-285ff7fbe007 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:33.489118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838d43c-60aa-4ddf-9ea8-3bb2c76ee8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:33.490461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838d43c-60aa-4ddf-9ea8-3bb2c76ee8cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:33.490945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838d43c-60aa-4ddf-9ea8-3bb2c76ee8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:33.491697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838d43c-60aa-4ddf-9ea8-3bb2c76ee8cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:33.492913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838d43c-60aa-4ddf-9ea8-3bb2c76ee8cd None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=a06126b4287647ca9294ce703dd39691, user.id=864a584c698748e89d6245905b6f26fb) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:33.493429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838d43c-60aa-4ddf-9ea8-3bb2c76ee8cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:33.495823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838d43c-60aa-4ddf-9ea8-3bb2c76ee8cd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:33.504403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 102/205] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:33 2022] GET /identity/v3/role_assignments?user.id=864a584c698748e89d6245905b6f26fb&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:33.505519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838d43c-60aa-4ddf-9ea8-3bb2c76ee8cd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:33.540226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c9e9b-01a1-42ac-907f-e8a05aeaf1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:33.540851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c9e9b-01a1-42ac-907f-e8a05aeaf1de None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:33.541133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c9e9b-01a1-42ac-907f-e8a05aeaf1de None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:33.541407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c9e9b-01a1-42ac-907f-e8a05aeaf1de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:33.542174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c9e9b-01a1-42ac-907f-e8a05aeaf1de None admin] RBAC: Adding query filter params (role.id=9dbcf5e8a10a4728888b9f4e923f5814, scope.project.id=811b356a854c422f85cddd17b5c478aa, user.id=2cde2e98da504619a842cc2feb481e74) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:33.542504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c9e9b-01a1-42ac-907f-e8a05aeaf1de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:33.549118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c9e9b-01a1-42ac-907f-e8a05aeaf1de None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:33.566066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 104/206] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:33 2022] GET /identity/v3/role_assignments?user.id=2cde2e98da504619a842cc2feb481e74&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=9dbcf5e8a10a4728888b9f4e923f5814 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:33.660042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838d43c-60aa-4ddf-9ea8-3bb2c76ee8cd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:33.719422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d7522-7340-43cd-b31c-43478248364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:33.720474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d7522-7340-43cd-b31c-43478248364a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:33.721020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d7522-7340-43cd-b31c-43478248364a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:33.721493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d7522-7340-43cd-b31c-43478248364a None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:33.728803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d7522-7340-43cd-b31c-43478248364a None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:33.730716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d7522-7340-43cd-b31c-43478248364a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:33.732812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 103/207] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:33 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:33.739215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c9e9b-01a1-42ac-907f-e8a05aeaf1de None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:33.878590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c9e9b-01a1-42ac-907f-e8a05aeaf1de None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:33.916285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc310454-614d-419e-b972-b7117750d83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:33.917296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc310454-614d-419e-b972-b7117750d83a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:33.917774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc310454-614d-419e-b972-b7117750d83a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:33.918243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc310454-614d-419e-b972-b7117750d83a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:33.919192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc310454-614d-419e-b972-b7117750d83a None admin] RBAC: Adding query filter params (role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=a06126b4287647ca9294ce703dd39691, user.id=864a584c698748e89d6245905b6f26fb) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:33.919697 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc310454-614d-419e-b972-b7117750d83a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:33.921378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc310454-614d-419e-b972-b7117750d83a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:33.927769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 105/208] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:33 2022] GET /identity/v3/role_assignments?user.id=864a584c698748e89d6245905b6f26fb&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:35.314560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e99026-ab35-4485-9449-45894f0dd3ff None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:35.315163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e99026-ab35-4485-9449-45894f0dd3ff None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:35.315163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e99026-ab35-4485-9449-45894f0dd3ff None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:35.316844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 104/209] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:35.323072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a729350-d5cd-4fb3-b8ea-6e9f103128da None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:35.324631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a729350-d5cd-4fb3-b8ea-6e9f103128da None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:35.325130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a729350-d5cd-4fb3-b8ea-6e9f103128da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:35.345397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5a729350-d5cd-4fb3-b8ea-6e9f103128da None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:35.346947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5a729350-d5cd-4fb3-b8ea-6e9f103128da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:35.365701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:35.424071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 106/210] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:35.438575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9a033-7258-4e75-b91b-24b572080105 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:35.438864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9a033-7258-4e75-b91b-24b572080105 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:35.439150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9a033-7258-4e75-b91b-24b572080105 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:35.440615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 105/211] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:35.445799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55644aa-2a38-4ac3-8541-ad80c679d0a3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:35.446093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55644aa-2a38-4ac3-8541-ad80c679d0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:35.446372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55644aa-2a38-4ac3-8541-ad80c679d0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:35.466297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c55644aa-2a38-4ac3-8541-ad80c679d0a3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:35.467500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c55644aa-2a38-4ac3-8541-ad80c679d0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:35.537536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 107/212] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:35.701497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9a033-7258-4e75-b91b-24b572080105 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:35.704571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:35.825507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9a033-7258-4e75-b91b-24b572080105 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:35.838818 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55644aa-2a38-4ac3-8541-ad80c679d0a3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:35.867014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222313e-7f86-4c82-99f2-8ecfcc0b6e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:35.867892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222313e-7f86-4c82-99f2-8ecfcc0b6e65 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:35.868178 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222313e-7f86-4c82-99f2-8ecfcc0b6e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:35.868457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222313e-7f86-4c82-99f2-8ecfcc0b6e65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:35.880406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222313e-7f86-4c82-99f2-8ecfcc0b6e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3222313e-7f86-4c82-99f2-8ecfcc0b6e65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:35.887401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:35.888536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:35.889309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 106/213] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:35.893101 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c93da-3d00-4fc0-be8d-310ed3f3d7be None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:35.893381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c93da-3d00-4fc0-be8d-310ed3f3d7be None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:35.893657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c93da-3d00-4fc0-be8d-310ed3f3d7be None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:35.895076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 107/214] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:35.895825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c93da-3d00-4fc0-be8d-310ed3f3d7be None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:35.933616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:35.955914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55644aa-2a38-4ac3-8541-ad80c679d0a3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:35.973980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:35.993840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933a6e0-6117-4553-a6cd-cf6ad16f6082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:35.994472 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933a6e0-6117-4553-a6cd-cf6ad16f6082 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:35.994750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933a6e0-6117-4553-a6cd-cf6ad16f6082 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:35.995018 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933a6e0-6117-4553-a6cd-cf6ad16f6082 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:35.996200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933a6e0-6117-4553-a6cd-cf6ad16f6082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:35.997633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933a6e0-6117-4553-a6cd-cf6ad16f6082 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:35.999382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 108/215] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:36.000391 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc3e3-a788-46f9-a98d-9f723667a330 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.000670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc3e3-a788-46f9-a98d-9f723667a330 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.000936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dc3e3-a788-46f9-a98d-9f723667a330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.018993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3e6dc3e3-a788-46f9-a98d-9f723667a330 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:36.020195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3e6dc3e3-a788-46f9-a98d-9f723667a330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:36.020445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c93da-3d00-4fc0-be8d-310ed3f3d7be None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:36.059660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdde494-1bc2-44a7-943f-6c47dd86fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:36.060683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdde494-1bc2-44a7-943f-6c47dd86fb36 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.060965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdde494-1bc2-44a7-943f-6c47dd86fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.061228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdde494-1bc2-44a7-943f-6c47dd86fb36 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.061926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdde494-1bc2-44a7-943f-6c47dd86fb36 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:36.062233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdde494-1bc2-44a7-943f-6c47dd86fb36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:36.063632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdde494-1bc2-44a7-943f-6c47dd86fb36 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:36.073243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 108/216] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:36.074367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdde494-1bc2-44a7-943f-6c47dd86fb36 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:36.103606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 109/217] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:36.105425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc3e3-a788-46f9-a98d-9f723667a330 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:36.202299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:36.208213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdde494-1bc2-44a7-943f-6c47dd86fb36 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:36.231873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dc3e3-a788-46f9-a98d-9f723667a330 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:36.240766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:36.253168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42c041-acf4-4820-8961-cbb9e819f6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:36.253777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42c041-acf4-4820-8961-cbb9e819f6ab None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.254055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42c041-acf4-4820-8961-cbb9e819f6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.254325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42c041-acf4-4820-8961-cbb9e819f6ab None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.254975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42c041-acf4-4820-8961-cbb9e819f6ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:36.256449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42c041-acf4-4820-8961-cbb9e819f6ab None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:36.272220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756957e7-8e15-4dd9-a03a-d888d2469307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:36.272909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756957e7-8e15-4dd9-a03a-d888d2469307 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.273330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756957e7-8e15-4dd9-a03a-d888d2469307 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.273684 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756957e7-8e15-4dd9-a03a-d888d2469307 None admin] PATH_INFO: `/v3/users/15fe769a5ea24b4d8bb81a2353520127` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.284056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 109/218] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:06:36 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:36.284056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf0487-4851-4478-9983-7a3271538237 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.284056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf0487-4851-4478-9983-7a3271538237 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.284056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf0487-4851-4478-9983-7a3271538237 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.284056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 110/219] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:36.288385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff311b-f2e0-4c81-9018-c591e63bcb24 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.288672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff311b-f2e0-4c81-9018-c591e63bcb24 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.288954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff311b-f2e0-4c81-9018-c591e63bcb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.295215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756957e7-8e15-4dd9-a03a-d888d2469307 None admin] RBAC: Authorizing `identity:get_user(user_id=15fe769a5ea24b4d8bb81a2353520127)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:36.297658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756957e7-8e15-4dd9-a03a-d888d2469307 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:36.299783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 110/220] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:36 2022] GET /identity/v3/users/15fe769a5ea24b4d8bb81a2353520127 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:36.305513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756957e7-8e15-4dd9-a03a-d888d2469307 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:36.322631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c0ff311b-f2e0-4c81-9018-c591e63bcb24 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:36.324575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c0ff311b-f2e0-4c81-9018-c591e63bcb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:36.415503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 111/221] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:36.416446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff311b-f2e0-4c81-9018-c591e63bcb24 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:36.439403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756957e7-8e15-4dd9-a03a-d888d2469307 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:36.487951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f33008-0cc3-4bb5-bd92-fdfc8bb4413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:36.488613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f33008-0cc3-4bb5-bd92-fdfc8bb4413a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.488800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f33008-0cc3-4bb5-bd92-fdfc8bb4413a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.489601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f33008-0cc3-4bb5-bd92-fdfc8bb4413a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.500952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:36.560063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff311b-f2e0-4c81-9018-c591e63bcb24 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:36.576473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f33008-0cc3-4bb5-bd92-fdfc8bb4413a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:36.578530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f33008-0cc3-4bb5-bd92-fdfc8bb4413a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:36.580108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:36.613727 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdf591-e41a-411a-9f1f-9ccd37bafe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:36.614372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdf591-e41a-411a-9f1f-9ccd37bafe52 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.614664 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdf591-e41a-411a-9f1f-9ccd37bafe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.614915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdf591-e41a-411a-9f1f-9ccd37bafe52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.629009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdf591-e41a-411a-9f1f-9ccd37bafe52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-82fdf591-e41a-411a-9f1f-9ccd37bafe52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:36.636683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:36.637591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:36.638373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:36.639495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 112/222] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:06:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:36.644714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdf591-e41a-411a-9f1f-9ccd37bafe52 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:36.719086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 111/223] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 12:06:36.721014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f33008-0cc3-4bb5-bd92-fdfc8bb4413a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:36.767708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:36.790792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdf591-e41a-411a-9f1f-9ccd37bafe52 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:36.832166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaae033-feef-46ad-a9e5-136b234ec40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:36.833181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaae033-feef-46ad-a9e5-136b234ec40e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.833672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaae033-feef-46ad-a9e5-136b234ec40e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.834180 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaae033-feef-46ad-a9e5-136b234ec40e None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.835067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaae033-feef-46ad-a9e5-136b234ec40e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:36.835595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaae033-feef-46ad-a9e5-136b234ec40e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:36.837348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaae033-feef-46ad-a9e5-136b234ec40e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:36.846607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f33008-0cc3-4bb5-bd92-fdfc8bb4413a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:36.847463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 113/224] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:36.852964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaae033-feef-46ad-a9e5-136b234ec40e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:36.892804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:36.893558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05beed-94b1-4d81-b3ba-9d38ea10124c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:36.894741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05beed-94b1-4d81-b3ba-9d38ea10124c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:36.895211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05beed-94b1-4d81-b3ba-9d38ea10124c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:36.895657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05beed-94b1-4d81-b3ba-9d38ea10124c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:36.907150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05beed-94b1-4d81-b3ba-9d38ea10124c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2b05beed-94b1-4d81-b3ba-9d38ea10124c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:36.913299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:36.914151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:36.914934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:36.922325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 112/225] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:06:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:36.923878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05beed-94b1-4d81-b3ba-9d38ea10124c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:36.971622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaae033-feef-46ad-a9e5-136b234ec40e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:37.008217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca42174-8582-4dd9-9768-bc1118d74fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:37.008852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42174-8582-4dd9-9768-bc1118d74fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:37.009134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42174-8582-4dd9-9768-bc1118d74fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:37.009452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42174-8582-4dd9-9768-bc1118d74fb1 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:37.011743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca42174-8582-4dd9-9768-bc1118d74fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:37.012797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca42174-8582-4dd9-9768-bc1118d74fb1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:37.014582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 114/226] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:36 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:37.015864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca42174-8582-4dd9-9768-bc1118d74fb1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:37.049918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:37.054185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05beed-94b1-4d81-b3ba-9d38ea10124c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:37.097765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd313c0-648e-4544-b868-2d1c8bf7bfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:37.098410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd313c0-648e-4544-b868-2d1c8bf7bfad None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:37.098736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd313c0-648e-4544-b868-2d1c8bf7bfad None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:37.099013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd313c0-648e-4544-b868-2d1c8bf7bfad None admin] PATH_INFO: `/v3/users/864a584c698748e89d6245905b6f26fb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:37.100188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd313c0-648e-4544-b868-2d1c8bf7bfad None admin] RBAC: Authorizing `identity:get_user(user_id=864a584c698748e89d6245905b6f26fb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:37.101641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd313c0-648e-4544-b868-2d1c8bf7bfad None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:37.103421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 113/227] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:36 2022] GET /identity/v3/users/864a584c698748e89d6245905b6f26fb => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:37.109366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd313c0-648e-4544-b868-2d1c8bf7bfad None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:37.128370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:37.133143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca42174-8582-4dd9-9768-bc1118d74fb1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:37.170278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ed3b5-630d-413d-b3fe-41e16aa0cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:37.171227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ed3b5-630d-413d-b3fe-41e16aa0cb9d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:37.171720 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ed3b5-630d-413d-b3fe-41e16aa0cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:37.172205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ed3b5-630d-413d-b3fe-41e16aa0cb9d None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:37.173104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ed3b5-630d-413d-b3fe-41e16aa0cb9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:37.173584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ed3b5-630d-413d-b3fe-41e16aa0cb9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:37.175181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ed3b5-630d-413d-b3fe-41e16aa0cb9d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:37.184538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 115/228] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:06:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:37.185686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ed3b5-630d-413d-b3fe-41e16aa0cb9d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:37.231316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd313c0-648e-4544-b868-2d1c8bf7bfad None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:37.282766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32c232-1909-42ac-94ae-4f649734bc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:37.283681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32c232-1909-42ac-94ae-4f649734bc51 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:37.284166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32c232-1909-42ac-94ae-4f649734bc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:37.284614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32c232-1909-42ac-94ae-4f649734bc51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:37.285521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32c232-1909-42ac-94ae-4f649734bc51 None admin] RBAC: Adding query filter params (role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=a06126b4287647ca9294ce703dd39691, user.id=15fe769a5ea24b4d8bb81a2353520127) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:37.286006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32c232-1909-42ac-94ae-4f649734bc51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:37.287634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32c232-1909-42ac-94ae-4f649734bc51 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:37.299911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 114/229] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:37 2022] GET /identity/v3/role_assignments?user.id=15fe769a5ea24b4d8bb81a2353520127&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:37.301198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32c232-1909-42ac-94ae-4f649734bc51 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:37.307290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ed3b5-630d-413d-b3fe-41e16aa0cb9d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:37.350602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:37.360486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da606946-ba4b-48f3-9145-077a256749ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:37.361864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da606946-ba4b-48f3-9145-077a256749ab None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:37.364226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da606946-ba4b-48f3-9145-077a256749ab None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:37.365124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da606946-ba4b-48f3-9145-077a256749ab None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:37.371785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da606946-ba4b-48f3-9145-077a256749ab None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:37.373641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da606946-ba4b-48f3-9145-077a256749ab None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:37.375633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 116/230] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:37 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:37.385369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da606946-ba4b-48f3-9145-077a256749ab None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:37.457204 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:37.515366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32c232-1909-42ac-94ae-4f649734bc51 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:37.553014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3d557-5503-4fff-8cbd-bf2dd72878ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:37.553961 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3d557-5503-4fff-8cbd-bf2dd72878ea None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:37.554420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3d557-5503-4fff-8cbd-bf2dd72878ea None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:37.554870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3d557-5503-4fff-8cbd-bf2dd72878ea None admin] PATH_INFO: `/v3/users/864a584c698748e89d6245905b6f26fb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:37.556320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3d557-5503-4fff-8cbd-bf2dd72878ea None admin] RBAC: Authorizing `identity:get_user(user_id=864a584c698748e89d6245905b6f26fb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:37.558553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d3d557-5503-4fff-8cbd-bf2dd72878ea None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:37.560371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 115/231] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:37 2022] GET /identity/v3/users/864a584c698748e89d6245905b6f26fb => generated 283 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:37.567346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3d557-5503-4fff-8cbd-bf2dd72878ea None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:37.574008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da606946-ba4b-48f3-9145-077a256749ab None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:37.612173 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dada3-3085-4a47-a12f-246b2e20e2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:37.612780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dada3-3085-4a47-a12f-246b2e20e2c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:37.613056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dada3-3085-4a47-a12f-246b2e20e2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:37.613332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dada3-3085-4a47-a12f-246b2e20e2c5 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691/users/864a584c698748e89d6245905b6f26fb/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:37.615247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7dada3-3085-4a47-a12f-246b2e20e2c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a06126b4287647ca9294ce703dd39691, user_id=864a584c698748e89d6245905b6f26fb, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:37.616732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7dada3-3085-4a47-a12f-246b2e20e2c5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:37.633979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 117/232] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:06:37 2022] PUT /identity/v3/projects/a06126b4287647ca9294ce703dd39691/users/864a584c698748e89d6245905b6f26fb/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:37.730849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:37.735203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3d557-5503-4fff-8cbd-bf2dd72878ea None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:37.781306 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd11be-7303-4e75-9ef6-d2b20f3b078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:37.781963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd11be-7303-4e75-9ef6-d2b20f3b078d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:37.782246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd11be-7303-4e75-9ef6-d2b20f3b078d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:37.782521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd11be-7303-4e75-9ef6-d2b20f3b078d None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:37.783772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd11be-7303-4e75-9ef6-d2b20f3b078d None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:37.785445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd11be-7303-4e75-9ef6-d2b20f3b078d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:37.787699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 116/233] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:37 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:37.793393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dada3-3085-4a47-a12f-246b2e20e2c5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:37.928143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:37.938762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dada3-3085-4a47-a12f-246b2e20e2c5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:37.978497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f780728-4731-4b04-9ecc-47eb7b066879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:37.978497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f780728-4731-4b04-9ecc-47eb7b066879 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:37.978497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f780728-4731-4b04-9ecc-47eb7b066879 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:37.978497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f780728-4731-4b04-9ecc-47eb7b066879 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691/users/864a584c698748e89d6245905b6f26fb/roles/23d9f4769fd6424bae83090a691c7f94` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:37.984814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f780728-4731-4b04-9ecc-47eb7b066879 None admin] RBAC: Authorizing `identity:create_grant(project_id=a06126b4287647ca9294ce703dd39691, user_id=864a584c698748e89d6245905b6f26fb, role_id=23d9f4769fd6424bae83090a691c7f94)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:37.985082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f780728-4731-4b04-9ecc-47eb7b066879 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:38.003323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 118/234] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:06:37 2022] PUT /identity/v3/projects/a06126b4287647ca9294ce703dd39691/users/864a584c698748e89d6245905b6f26fb/roles/23d9f4769fd6424bae83090a691c7f94 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:38.942943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb5315-9062-4a1f-9075-fe0e3abd1678 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:38.944301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb5315-9062-4a1f-9075-fe0e3abd1678 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:38.944792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb5315-9062-4a1f-9075-fe0e3abd1678 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:38.945440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273e8cb-f824-45f5-88fc-5561c633f633 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:38.945859 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273e8cb-f824-45f5-88fc-5561c633f633 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:38.946020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273e8cb-f824-45f5-88fc-5561c633f633 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:38.947601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 119/236] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:38.947837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 117/236] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:38.953143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfb687-f6c7-4686-a273-bfa8d479504c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:38.953656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfb687-f6c7-4686-a273-bfa8d479504c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:38.953760 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfb687-f6c7-4686-a273-bfa8d479504c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:38.955248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a8d81-6e65-4a0c-8df3-95b46add2209 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:38.955742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a8d81-6e65-4a0c-8df3-95b46add2209 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:38.956262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a8d81-6e65-4a0c-8df3-95b46add2209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:38.975782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-24cfb687-f6c7-4686-a273-bfa8d479504c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:38.975782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-24cfb687-f6c7-4686-a273-bfa8d479504c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:38.988781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0e9a8d81-6e65-4a0c-8df3-95b46add2209 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:38.990320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0e9a8d81-6e65-4a0c-8df3-95b46add2209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:39.052296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 120/237] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:39.079309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 118/238] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:39.359414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfb687-f6c7-4686-a273-bfa8d479504c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:39.479172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:39.534750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfb687-f6c7-4686-a273-bfa8d479504c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:39.544915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a8d81-6e65-4a0c-8df3-95b46add2209 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:39.687864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e99c5-43b4-4bb1-94c0-1b1fedd906cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:39.687864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e99c5-43b4-4bb1-94c0-1b1fedd906cb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:39.687864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e99c5-43b4-4bb1-94c0-1b1fedd906cb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:39.687864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e99c5-43b4-4bb1-94c0-1b1fedd906cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:39.688475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e99c5-43b4-4bb1-94c0-1b1fedd906cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:39.690146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e99c5-43b4-4bb1-94c0-1b1fedd906cb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:39.698765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 121/239] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:39.698765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e99c5-43b4-4bb1-94c0-1b1fedd906cb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:39.738427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:39.749775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a8d81-6e65-4a0c-8df3-95b46add2209 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:39.865485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e57774-8667-4197-a27c-3bfc4ac0f5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:39.866127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e57774-8667-4197-a27c-3bfc4ac0f5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:39.866403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e57774-8667-4197-a27c-3bfc4ac0f5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:39.867792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e57774-8667-4197-a27c-3bfc4ac0f5e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:39.879743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e57774-8667-4197-a27c-3bfc4ac0f5e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-90e57774-8667-4197-a27c-3bfc4ac0f5e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:39.896903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:39.897823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:39.898592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 119/240] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:39.908769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e57774-8667-4197-a27c-3bfc4ac0f5e3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:40.039667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e99c5-43b4-4bb1-94c0-1b1fedd906cb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:40.132747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d509f82-b4a8-4cf5-bad2-6823b40f8c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:40.133369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d509f82-b4a8-4cf5-bad2-6823b40f8c19 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.133646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d509f82-b4a8-4cf5-bad2-6823b40f8c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.133981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d509f82-b4a8-4cf5-bad2-6823b40f8c19 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.134646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d509f82-b4a8-4cf5-bad2-6823b40f8c19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:40.137666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d509f82-b4a8-4cf5-bad2-6823b40f8c19 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:40.173934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 122/241] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:06:39 2022] POST /identity/v3/users => generated 293 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:40.175243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f332097-6756-4d2b-bb89-5ffecccf56a5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.175629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f332097-6756-4d2b-bb89-5ffecccf56a5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.176106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f332097-6756-4d2b-bb89-5ffecccf56a5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.181808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 123/242] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:40.183333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d132ab8-1e15-4ef1-92ec-d67080f21352 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.183619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d132ab8-1e15-4ef1-92ec-d67080f21352 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.183985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d132ab8-1e15-4ef1-92ec-d67080f21352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.184414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e57774-8667-4197-a27c-3bfc4ac0f5e3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:40.192372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:40.237326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2d132ab8-1e15-4ef1-92ec-d67080f21352 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:40.239686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2d132ab8-1e15-4ef1-92ec-d67080f21352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:40.244906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39369028-a897-4494-86e8-4adcf7f300c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:40.245621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39369028-a897-4494-86e8-4adcf7f300c9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.245910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39369028-a897-4494-86e8-4adcf7f300c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.246233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39369028-a897-4494-86e8-4adcf7f300c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.247062 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39369028-a897-4494-86e8-4adcf7f300c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:40.247369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39369028-a897-4494-86e8-4adcf7f300c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:40.248820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39369028-a897-4494-86e8-4adcf7f300c9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:40.275492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 120/243] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:40.281861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39369028-a897-4494-86e8-4adcf7f300c9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:40.319031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:40.393186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 124/244] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:40.449865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39369028-a897-4494-86e8-4adcf7f300c9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:40.498038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24b9cf-aa97-4703-8548-1d4bdb358f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:40.498701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24b9cf-aa97-4703-8548-1d4bdb358f9a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.498981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24b9cf-aa97-4703-8548-1d4bdb358f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.499258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24b9cf-aa97-4703-8548-1d4bdb358f9a None admin] PATH_INFO: `/v3/users/e2d6fab82d444951bb8dd691f084f29c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.514635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24b9cf-aa97-4703-8548-1d4bdb358f9a None admin] RBAC: Authorizing `identity:get_user(user_id=e2d6fab82d444951bb8dd691f084f29c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:40.516134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24b9cf-aa97-4703-8548-1d4bdb358f9a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:40.518183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 121/245] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:40 2022] GET /identity/v3/users/e2d6fab82d444951bb8dd691f084f29c => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:40.523347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d132ab8-1e15-4ef1-92ec-d67080f21352 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:40.614386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7cf1-ef38-44a6-a8c3-02ffc461733e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.614619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7cf1-ef38-44a6-a8c3-02ffc461733e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.615022 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf7cf1-ef38-44a6-a8c3-02ffc461733e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.616694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 122/246] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:40.623148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92719681-6243-4082-91ea-870c00315d0c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.623437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92719681-6243-4082-91ea-870c00315d0c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.623714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92719681-6243-4082-91ea-870c00315d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.647443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-92719681-6243-4082-91ea-870c00315d0c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:40.648674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-92719681-6243-4082-91ea-870c00315d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:40.663785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d132ab8-1e15-4ef1-92ec-d67080f21352 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:40.704670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4850ef-9630-4e4b-be0f-41f1b87f8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:40.705287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4850ef-9630-4e4b-be0f-41f1b87f8684 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.705565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4850ef-9630-4e4b-be0f-41f1b87f8684 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.705834 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4850ef-9630-4e4b-be0f-41f1b87f8684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.709635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:40.738685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 123/247] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:40.740010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92719681-6243-4082-91ea-870c00315d0c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:40.757130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:40.815125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4850ef-9630-4e4b-be0f-41f1b87f8684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:40.817026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4850ef-9630-4e4b-be0f-41f1b87f8684 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:40.867328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92719681-6243-4082-91ea-870c00315d0c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:40.918672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bba66e-46d0-4cf9-8075-1bb30ef3dffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:40.919322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bba66e-46d0-4cf9-8075-1bb30ef3dffc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.919603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bba66e-46d0-4cf9-8075-1bb30ef3dffc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.919965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bba66e-46d0-4cf9-8075-1bb30ef3dffc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.936084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bba66e-46d0-4cf9-8075-1bb30ef3dffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d8bba66e-46d0-4cf9-8075-1bb30ef3dffc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:40.941598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:40.942485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:40.943246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 124/248] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:40.946432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242aafb-7264-49ff-8db6-07ab65583bad None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.947096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242aafb-7264-49ff-8db6-07ab65583bad None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.949227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242aafb-7264-49ff-8db6-07ab65583bad None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:40.950687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 125/249] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:40.951234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242aafb-7264-49ff-8db6-07ab65583bad None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:40.963900 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 125/250] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:40.965170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac5fab-cd41-422d-8529-2b249acd7284 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:40.965456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac5fab-cd41-422d-8529-2b249acd7284 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:40.965984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac5fab-cd41-422d-8529-2b249acd7284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:41.012850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-22ac5fab-cd41-422d-8529-2b249acd7284 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:41.014105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22ac5fab-cd41-422d-8529-2b249acd7284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:41.033047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:41.077829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242aafb-7264-49ff-8db6-07ab65583bad None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:41.080924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:41.116466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d563e1e-1f2f-45b2-8237-39e38c8155c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:41.117220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d563e1e-1f2f-45b2-8237-39e38c8155c8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:41.117639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d563e1e-1f2f-45b2-8237-39e38c8155c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:41.117926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d563e1e-1f2f-45b2-8237-39e38c8155c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:41.118667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d563e1e-1f2f-45b2-8237-39e38c8155c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:41.119031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d563e1e-1f2f-45b2-8237-39e38c8155c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:41.121087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d563e1e-1f2f-45b2-8237-39e38c8155c8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:41.130077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 126/251] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:41.132625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d563e1e-1f2f-45b2-8237-39e38c8155c8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:41.136603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 126/252] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:41.137688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac5fab-cd41-422d-8529-2b249acd7284 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:41.241978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d563e1e-1f2f-45b2-8237-39e38c8155c8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:41.270909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:41.279375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f22c21-e4a2-4906-8ce5-4465adac34a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:41.280142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f22c21-e4a2-4906-8ce5-4465adac34a5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:41.280500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f22c21-e4a2-4906-8ce5-4465adac34a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:41.280881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f22c21-e4a2-4906-8ce5-4465adac34a5 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:41.282118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f22c21-e4a2-4906-8ce5-4465adac34a5 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:41.283700 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f22c21-e4a2-4906-8ce5-4465adac34a5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:41.285631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 127/253] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:41 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:41.286649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f22c21-e4a2-4906-8ce5-4465adac34a5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:41.298617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac5fab-cd41-422d-8529-2b249acd7284 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:41.384502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf14863-0fb9-424c-9c8e-d1ee9a498320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:41.385567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf14863-0fb9-424c-9c8e-d1ee9a498320 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:41.386799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf14863-0fb9-424c-9c8e-d1ee9a498320 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:41.387260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf14863-0fb9-424c-9c8e-d1ee9a498320 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:41.395646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f22c21-e4a2-4906-8ce5-4465adac34a5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:41.417662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf14863-0fb9-424c-9c8e-d1ee9a498320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2cf14863-0fb9-424c-9c8e-d1ee9a498320 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:41.426503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:41.427382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:41.428411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:41.433151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 127/254] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:06:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:41.435278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf14863-0fb9-424c-9c8e-d1ee9a498320 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:41.466390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c33a6-2f3c-44bc-b070-44df52a96f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:41.467269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c33a6-2f3c-44bc-b070-44df52a96f5c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:41.468441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c33a6-2f3c-44bc-b070-44df52a96f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:41.468723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c33a6-2f3c-44bc-b070-44df52a96f5c None admin] PATH_INFO: `/v3/users/15fe769a5ea24b4d8bb81a2353520127` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:41.470005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c33a6-2f3c-44bc-b070-44df52a96f5c None admin] RBAC: Authorizing `identity:get_user(user_id=15fe769a5ea24b4d8bb81a2353520127)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:41.471442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c33a6-2f3c-44bc-b070-44df52a96f5c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:41.480196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 128/255] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:41 2022] GET /identity/v3/users/15fe769a5ea24b4d8bb81a2353520127 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:41.481413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c33a6-2f3c-44bc-b070-44df52a96f5c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:41.496225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:41.576829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:41.609605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf14863-0fb9-424c-9c8e-d1ee9a498320 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:41.627019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c33a6-2f3c-44bc-b070-44df52a96f5c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:41.656875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d466fe-08e8-4e4f-a9ea-a071840ea8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:41.658484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d466fe-08e8-4e4f-a9ea-a071840ea8bf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:41.659030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d466fe-08e8-4e4f-a9ea-a071840ea8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:41.659451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d466fe-08e8-4e4f-a9ea-a071840ea8bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:41.660709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d466fe-08e8-4e4f-a9ea-a071840ea8bf None admin] RBAC: Adding query filter params (role.id=9dbcf5e8a10a4728888b9f4e923f5814, scope.project.id=a06126b4287647ca9294ce703dd39691, user.id=e2d6fab82d444951bb8dd691f084f29c) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:41.661691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d466fe-08e8-4e4f-a9ea-a071840ea8bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:41.663826 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d466fe-08e8-4e4f-a9ea-a071840ea8bf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:41.667715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b860ca5-d186-4174-876c-f014c4ebaec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:41.669176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b860ca5-d186-4174-876c-f014c4ebaec2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:41.669565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b860ca5-d186-4174-876c-f014c4ebaec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:41.670243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b860ca5-d186-4174-876c-f014c4ebaec2 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:41.671077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b860ca5-d186-4174-876c-f014c4ebaec2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:41.671495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b860ca5-d186-4174-876c-f014c4ebaec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:41.672307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 128/256] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:41 2022] GET /identity/v3/role_assignments?user.id=e2d6fab82d444951bb8dd691f084f29c&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=9dbcf5e8a10a4728888b9f4e923f5814 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:41.673506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d466fe-08e8-4e4f-a9ea-a071840ea8bf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:41.674329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b860ca5-d186-4174-876c-f014c4ebaec2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:41.686344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 129/257] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:41.692168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b860ca5-d186-4174-876c-f014c4ebaec2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:41.853325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:41.876106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b860ca5-d186-4174-876c-f014c4ebaec2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:41.879368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d466fe-08e8-4e4f-a9ea-a071840ea8bf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:41.922203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:41.939950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa23d7e-c6c6-4fce-9fbe-7982dfcc3e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:41.940599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa23d7e-c6c6-4fce-9fbe-7982dfcc3e86 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:41.940884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa23d7e-c6c6-4fce-9fbe-7982dfcc3e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:41.941157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa23d7e-c6c6-4fce-9fbe-7982dfcc3e86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:41.942291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470f999-d275-413d-8623-a737af756a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:41.942931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470f999-d275-413d-8623-a737af756a50 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:41.943210 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470f999-d275-413d-8623-a737af756a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:41.943479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6470f999-d275-413d-8623-a737af756a50 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:41.945483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470f999-d275-413d-8623-a737af756a50 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:41.946951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6470f999-d275-413d-8623-a737af756a50 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:41.949216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 129/258] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:41 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:41.949995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470f999-d275-413d-8623-a737af756a50 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:41.969504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa23d7e-c6c6-4fce-9fbe-7982dfcc3e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0aa23d7e-c6c6-4fce-9fbe-7982dfcc3e86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:41.974918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:41.975819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:41.976679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:41.981625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 130/259] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:06:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:41.982433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa23d7e-c6c6-4fce-9fbe-7982dfcc3e86 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:42.146024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6470f999-d275-413d-8623-a737af756a50 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:42.185788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d534fef-1d92-45b1-b585-77b69e000039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:42.186346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d534fef-1d92-45b1-b585-77b69e000039 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:42.186630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d534fef-1d92-45b1-b585-77b69e000039 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:42.186906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d534fef-1d92-45b1-b585-77b69e000039 None admin] PATH_INFO: `/v3/users/864a584c698748e89d6245905b6f26fb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:42.188147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d534fef-1d92-45b1-b585-77b69e000039 None admin] RBAC: Authorizing `identity:get_user(user_id=864a584c698748e89d6245905b6f26fb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:42.189603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d534fef-1d92-45b1-b585-77b69e000039 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:42.191417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 130/260] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:41 2022] GET /identity/v3/users/864a584c698748e89d6245905b6f26fb => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:42.192073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d534fef-1d92-45b1-b585-77b69e000039 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:42.195699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa23d7e-c6c6-4fce-9fbe-7982dfcc3e86 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:42.220818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:42.265819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dedfd-c753-469f-9ef5-e395c154e7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:42.266485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dedfd-c753-469f-9ef5-e395c154e7c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:42.266770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dedfd-c753-469f-9ef5-e395c154e7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:42.267046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318dedfd-c753-469f-9ef5-e395c154e7c3 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691/users/15fe769a5ea24b4d8bb81a2353520127/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:42.274422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318dedfd-c753-469f-9ef5-e395c154e7c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a06126b4287647ca9294ce703dd39691, user_id=15fe769a5ea24b4d8bb81a2353520127, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:42.275949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318dedfd-c753-469f-9ef5-e395c154e7c3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:42.297643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 131/261] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:06:41 2022] PUT /identity/v3/projects/a06126b4287647ca9294ce703dd39691/users/15fe769a5ea24b4d8bb81a2353520127/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:42.298483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dedfd-c753-469f-9ef5-e395c154e7c3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:42.324459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d534fef-1d92-45b1-b585-77b69e000039 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:42.386089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a7144-dcc4-463d-9a75-ca3a51cc6831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:42.386979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a7144-dcc4-463d-9a75-ca3a51cc6831 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:42.387395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a7144-dcc4-463d-9a75-ca3a51cc6831 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:42.387847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a7144-dcc4-463d-9a75-ca3a51cc6831 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:42.388701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a7144-dcc4-463d-9a75-ca3a51cc6831 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:42.389143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a7144-dcc4-463d-9a75-ca3a51cc6831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:42.390668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a7144-dcc4-463d-9a75-ca3a51cc6831 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:42.399551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 131/262] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:06:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:42.408907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a7144-dcc4-463d-9a75-ca3a51cc6831 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:42.414623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:42.533486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318dedfd-c753-469f-9ef5-e395c154e7c3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:42.572196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:42.580215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405567-5e10-4ca3-8c6a-1dda0ae37afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:42.581082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405567-5e10-4ca3-8c6a-1dda0ae37afd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:42.581564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405567-5e10-4ca3-8c6a-1dda0ae37afd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:42.581953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405567-5e10-4ca3-8c6a-1dda0ae37afd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:42.582483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a7144-dcc4-463d-9a75-ca3a51cc6831 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:42.626203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d7d6f-39ac-4205-92de-1e3f2d9eb09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:42.626846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d7d6f-39ac-4205-92de-1e3f2d9eb09f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:42.627128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d7d6f-39ac-4205-92de-1e3f2d9eb09f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:42.627405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d7d6f-39ac-4205-92de-1e3f2d9eb09f None admin] PATH_INFO: `/v3/users/864a584c698748e89d6245905b6f26fb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:42.628591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d7d6f-39ac-4205-92de-1e3f2d9eb09f None admin] RBAC: Authorizing `identity:get_user(user_id=864a584c698748e89d6245905b6f26fb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:42.630059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d7d6f-39ac-4205-92de-1e3f2d9eb09f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:42.632707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 132/263] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:42 2022] GET /identity/v3/users/864a584c698748e89d6245905b6f26fb => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:42.651380 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d7d6f-39ac-4205-92de-1e3f2d9eb09f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:42.685294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d405567-5e10-4ca3-8c6a-1dda0ae37afd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:42.687097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d405567-5e10-4ca3-8c6a-1dda0ae37afd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:42.737223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:42.798533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 132/264] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:42.804712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405567-5e10-4ca3-8c6a-1dda0ae37afd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:42.933833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d7d6f-39ac-4205-92de-1e3f2d9eb09f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:42.962973 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405567-5e10-4ca3-8c6a-1dda0ae37afd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:43.001949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b1ba6-0c3d-411b-8f06-4f5d2e192427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:43.002585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b1ba6-0c3d-411b-8f06-4f5d2e192427 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:43.002860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b1ba6-0c3d-411b-8f06-4f5d2e192427 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:43.003124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b1ba6-0c3d-411b-8f06-4f5d2e192427 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:43.006018 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b1ba6-0c3d-411b-8f06-4f5d2e192427 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:43.007475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b1ba6-0c3d-411b-8f06-4f5d2e192427 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:43.008409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef05f3-5c86-4137-9979-a106c9a2d32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:43.009012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef05f3-5c86-4137-9979-a106c9a2d32b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:43.009289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef05f3-5c86-4137-9979-a106c9a2d32b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:43.009555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef05f3-5c86-4137-9979-a106c9a2d32b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:43.013604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 133/265] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:42 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:43.024210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b1ba6-0c3d-411b-8f06-4f5d2e192427 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:43.040823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:43.062578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:43.187199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b1ba6-0c3d-411b-8f06-4f5d2e192427 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:43.193375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef05f3-5c86-4137-9979-a106c9a2d32b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:43.194297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef05f3-5c86-4137-9979-a106c9a2d32b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:43.256433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1948fd-2e1e-4538-882c-a705bd94e5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:43.257055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1948fd-2e1e-4538-882c-a705bd94e5ba None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:43.257335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1948fd-2e1e-4538-882c-a705bd94e5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:43.257607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1948fd-2e1e-4538-882c-a705bd94e5ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:43.258341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1948fd-2e1e-4538-882c-a705bd94e5ba None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=a06126b4287647ca9294ce703dd39691, user.id=864a584c698748e89d6245905b6f26fb) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:43.258647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1948fd-2e1e-4538-882c-a705bd94e5ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:43.260179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1948fd-2e1e-4538-882c-a705bd94e5ba None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:43.270701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 134/266] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:43 2022] GET /identity/v3/role_assignments?user.id=864a584c698748e89d6245905b6f26fb&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:43.372396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 133/267] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 721 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:43.378185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1948fd-2e1e-4538-882c-a705bd94e5ba None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:43.474285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c854a4-4815-4fca-99a4-1271b41f88e4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:43.474634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c854a4-4815-4fca-99a4-1271b41f88e4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:43.474856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c854a4-4815-4fca-99a4-1271b41f88e4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:43.476507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 134/268] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:43.485434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a8952-624f-4ab5-bee1-36c81e9e264a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:43.485744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a8952-624f-4ab5-bee1-36c81e9e264a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:43.485993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a8952-624f-4ab5-bee1-36c81e9e264a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:43.488193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:43.517465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1948fd-2e1e-4538-882c-a705bd94e5ba None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:43.527870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fa6a8952-624f-4ab5-bee1-36c81e9e264a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:43.529116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fa6a8952-624f-4ab5-bee1-36c81e9e264a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:43.596889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7887c-f29e-45fb-89bd-9308bca06762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:43.597982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7887c-f29e-45fb-89bd-9308bca06762 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:43.598457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7887c-f29e-45fb-89bd-9308bca06762 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:43.598925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7887c-f29e-45fb-89bd-9308bca06762 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:43.600576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7887c-f29e-45fb-89bd-9308bca06762 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:43.602411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7887c-f29e-45fb-89bd-9308bca06762 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:43.604448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 135/269] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:43 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:43.610853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7887c-f29e-45fb-89bd-9308bca06762 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:43.628816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:43.672096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 135/270] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:43.717692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa9873-4195-4505-9aff-a966984f9606 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:43.718023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa9873-4195-4505-9aff-a966984f9606 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:43.718241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa9873-4195-4505-9aff-a966984f9606 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:43.719728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 136/271] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:43.726452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dba531-d389-4208-9ac2-94452fb29fd4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:43.726730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dba531-d389-4208-9ac2-94452fb29fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:43.727004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dba531-d389-4208-9ac2-94452fb29fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:43.765739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-40dba531-d389-4208-9ac2-94452fb29fd4 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:43.768273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-40dba531-d389-4208-9ac2-94452fb29fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:43.788598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7887c-f29e-45fb-89bd-9308bca06762 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:43.872114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 137/272] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:43.888614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26162456-fb36-4122-8225-843fb771b3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:43.889560 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26162456-fb36-4122-8225-843fb771b3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:43.890059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26162456-fb36-4122-8225-843fb771b3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:43.890527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26162456-fb36-4122-8225-843fb771b3a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:43.891465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26162456-fb36-4122-8225-843fb771b3a6 None admin] RBAC: Adding query filter params (role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=a06126b4287647ca9294ce703dd39691, user.id=864a584c698748e89d6245905b6f26fb) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:43.892016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26162456-fb36-4122-8225-843fb771b3a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:43.893678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26162456-fb36-4122-8225-843fb771b3a6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:43.901159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 136/273] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:43 2022] GET /identity/v3/role_assignments?user.id=864a584c698748e89d6245905b6f26fb&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:44.029214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dba531-d389-4208-9ac2-94452fb29fd4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:44.142874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dba531-d389-4208-9ac2-94452fb29fd4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:44.158715 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:44.181860 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e5844-913a-4491-aedd-999a8088f712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:44.182509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e5844-913a-4491-aedd-999a8088f712 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:44.183338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e5844-913a-4491-aedd-999a8088f712 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:44.183607 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e5844-913a-4491-aedd-999a8088f712 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:44.184883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e5844-913a-4491-aedd-999a8088f712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:44.186358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e5844-913a-4491-aedd-999a8088f712 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:44.188528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 138/274] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:44.194236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26162456-fb36-4122-8225-843fb771b3a6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:44.200742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e5844-913a-4491-aedd-999a8088f712 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:44.219336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:44.336989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26162456-fb36-4122-8225-843fb771b3a6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:44.337618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e5844-913a-4491-aedd-999a8088f712 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:44.378890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767720f-6419-47d5-8aec-afb32f639806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:44.379511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767720f-6419-47d5-8aec-afb32f639806 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:44.379827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767720f-6419-47d5-8aec-afb32f639806 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:44.380117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767720f-6419-47d5-8aec-afb32f639806 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:44.380752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767720f-6419-47d5-8aec-afb32f639806 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:44.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767720f-6419-47d5-8aec-afb32f639806 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:44.388220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc579a0c-9bc3-4959-b891-c1bed884270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:44.389233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc579a0c-9bc3-4959-b891-c1bed884270c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:44.389705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc579a0c-9bc3-4959-b891-c1bed884270c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:44.390169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc579a0c-9bc3-4959-b891-c1bed884270c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:44.404517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc579a0c-9bc3-4959-b891-c1bed884270c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:44.406342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 137/275] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:06:44 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-cc579a0c-9bc3-4959-b891-c1bed884270c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:44.410157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:44.411191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:44.412042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:44.419426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 139/276] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:44.419604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767720f-6419-47d5-8aec-afb32f639806 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:44.469422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdb567-f53f-4667-b9a6-180a3347e436 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:44.469794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdb567-f53f-4667-b9a6-180a3347e436 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:44.470565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdb567-f53f-4667-b9a6-180a3347e436 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:44.472201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 140/277] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:44.478016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201111e-a29b-44f6-a098-6538c0c4e52b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:44.478324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201111e-a29b-44f6-a098-6538c0c4e52b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:44.478576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201111e-a29b-44f6-a098-6538c0c4e52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:44.503005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6201111e-a29b-44f6-a098-6538c0c4e52b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:44.504284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6201111e-a29b-44f6-a098-6538c0c4e52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:44.507459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:44.553340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767720f-6419-47d5-8aec-afb32f639806 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:44.569782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:44.593268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de2eee-972d-4abd-a07a-7a81fbf582b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:44.593889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de2eee-972d-4abd-a07a-7a81fbf582b0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:44.594165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de2eee-972d-4abd-a07a-7a81fbf582b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:44.594545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de2eee-972d-4abd-a07a-7a81fbf582b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:44.595323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de2eee-972d-4abd-a07a-7a81fbf582b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:44.595625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de2eee-972d-4abd-a07a-7a81fbf582b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:44.597045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de2eee-972d-4abd-a07a-7a81fbf582b0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:44.600170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 141/278] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:44.610864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 138/279] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:44.619996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201111e-a29b-44f6-a098-6538c0c4e52b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:44.733332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6201111e-a29b-44f6-a098-6538c0c4e52b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:44.781749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca7e17-1038-45eb-8219-6d8d1beb7a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:44.782725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca7e17-1038-45eb-8219-6d8d1beb7a1a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:44.783202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca7e17-1038-45eb-8219-6d8d1beb7a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:44.783669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca7e17-1038-45eb-8219-6d8d1beb7a1a None admin] PATH_INFO: `/v3/users/362ffed0b1804934b37ac5c4a93bc457` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:44.799613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca7e17-1038-45eb-8219-6d8d1beb7a1a None admin] RBAC: Authorizing `identity:get_user(user_id=362ffed0b1804934b37ac5c4a93bc457)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:44.801184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca7e17-1038-45eb-8219-6d8d1beb7a1a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:44.803017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 142/280] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:44 2022] GET /identity/v3/users/362ffed0b1804934b37ac5c4a93bc457 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:44.808759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de2eee-972d-4abd-a07a-7a81fbf582b0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:44.895382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca7e17-1038-45eb-8219-6d8d1beb7a1a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:44.931421 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:44.991691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de2eee-972d-4abd-a07a-7a81fbf582b0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:45.035221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca7e17-1038-45eb-8219-6d8d1beb7a1a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:45.048574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:45.061261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165581b-8a66-4693-b9ae-4c09af5fe7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:45.062656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165581b-8a66-4693-b9ae-4c09af5fe7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:45.063173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165581b-8a66-4693-b9ae-4c09af5fe7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:45.063449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165581b-8a66-4693-b9ae-4c09af5fe7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:45.065753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165581b-8a66-4693-b9ae-4c09af5fe7b4 None admin] RBAC: Adding query filter params (role.id=3367be0edc604df081b5ac7a9e2a7b92, user.id=362ffed0b1804934b37ac5c4a93bc457, scope.system=all) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:45.066064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165581b-8a66-4693-b9ae-4c09af5fe7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:45.067529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165581b-8a66-4693-b9ae-4c09af5fe7b4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:45.080856 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 139/281] 10.222.0.71 () {56 vars in 1308 bytes} [Wed Sep 21 12:06:44 2022] GET /identity/v3/role_assignments?user.id=362ffed0b1804934b37ac5c4a93bc457&scope.system=all&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:45.097603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d4c20-fdf4-4946-a45f-071244c6843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:45.098242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4c20-fdf4-4946-a45f-071244c6843f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:45.098520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4c20-fdf4-4946-a45f-071244c6843f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:45.098796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d4c20-fdf4-4946-a45f-071244c6843f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:45.121218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d4c20-fdf4-4946-a45f-071244c6843f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a69d4c20-fdf4-4946-a45f-071244c6843f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:45.126620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:45.127575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:45.128390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 143/282] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:06:45.134048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165581b-8a66-4693-b9ae-4c09af5fe7b4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:45.247504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165581b-8a66-4693-b9ae-4c09af5fe7b4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:45.288782 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d48d34-fbcf-40f3-8e8a-0ed2bb0b007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:45.289800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d48d34-fbcf-40f3-8e8a-0ed2bb0b007a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:45.290304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d48d34-fbcf-40f3-8e8a-0ed2bb0b007a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:45.290806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d48d34-fbcf-40f3-8e8a-0ed2bb0b007a None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:45.291696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d48d34-fbcf-40f3-8e8a-0ed2bb0b007a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:45.292310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d48d34-fbcf-40f3-8e8a-0ed2bb0b007a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:45.293933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d48d34-fbcf-40f3-8e8a-0ed2bb0b007a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:45.303631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 140/283] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:45.311791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d4c20-fdf4-4946-a45f-071244c6843f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:45.401452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec04ef-0e01-4e25-a500-629685bc4464 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:45.402406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec04ef-0e01-4e25-a500-629685bc4464 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:45.402877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec04ef-0e01-4e25-a500-629685bc4464 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:45.404723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 141/284] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:45.409969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efca45-f986-4bb4-b009-298b02aca447 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:45.410505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efca45-f986-4bb4-b009-298b02aca447 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:45.410978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22efca45-f986-4bb4-b009-298b02aca447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:45.435564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-22efca45-f986-4bb4-b009-298b02aca447 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:45.437143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-22efca45-f986-4bb4-b009-298b02aca447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:45.458297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:45.471705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d4c20-fdf4-4946-a45f-071244c6843f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:45.508372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b27dc2-eb53-4081-a61c-ec7847c81ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:45.508924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b27dc2-eb53-4081-a61c-ec7847c81ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:45.509199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b27dc2-eb53-4081-a61c-ec7847c81ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:45.509467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b27dc2-eb53-4081-a61c-ec7847c81ad1 None admin] PATH_INFO: `/v3/users/e2d6fab82d444951bb8dd691f084f29c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:45.511258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b27dc2-eb53-4081-a61c-ec7847c81ad1 None admin] RBAC: Authorizing `identity:get_user(user_id=e2d6fab82d444951bb8dd691f084f29c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:45.512697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b27dc2-eb53-4081-a61c-ec7847c81ad1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:45.514633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 144/285] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:45 2022] GET /identity/v3/users/e2d6fab82d444951bb8dd691f084f29c => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:45.521551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b27dc2-eb53-4081-a61c-ec7847c81ad1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:45.550724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:45.587139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 142/286] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:45.655769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b27dc2-eb53-4081-a61c-ec7847c81ad1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:45.690723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec81200-c71d-4cf6-bbb7-091e655a1a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:45.691358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec81200-c71d-4cf6-bbb7-091e655a1a7c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:45.691634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec81200-c71d-4cf6-bbb7-091e655a1a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:45.691936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec81200-c71d-4cf6-bbb7-091e655a1a7c None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:45.693058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec81200-c71d-4cf6-bbb7-091e655a1a7c None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:45.694512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec81200-c71d-4cf6-bbb7-091e655a1a7c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:45.696527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 145/287] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:45 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:45.704882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efca45-f986-4bb4-b009-298b02aca447 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:45.826207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22efca45-f986-4bb4-b009-298b02aca447 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:45.884531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4780d-6eb1-40f3-9d6f-a633af4c5d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:45.885213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4780d-6eb1-40f3-9d6f-a633af4c5d83 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:45.885489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4780d-6eb1-40f3-9d6f-a633af4c5d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:45.886301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4780d-6eb1-40f3-9d6f-a633af4c5d83 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691/users/e2d6fab82d444951bb8dd691f084f29c/roles/9dbcf5e8a10a4728888b9f4e923f5814` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:45.888436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4780d-6eb1-40f3-9d6f-a633af4c5d83 None admin] RBAC: Authorizing `identity:create_grant(project_id=a06126b4287647ca9294ce703dd39691, user_id=e2d6fab82d444951bb8dd691f084f29c, role_id=9dbcf5e8a10a4728888b9f4e923f5814)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:45.889898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4780d-6eb1-40f3-9d6f-a633af4c5d83 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:45.912248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 143/288] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:06:45 2022] PUT /identity/v3/projects/a06126b4287647ca9294ce703dd39691/users/e2d6fab82d444951bb8dd691f084f29c/roles/9dbcf5e8a10a4728888b9f4e923f5814 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:46.035519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec81200-c71d-4cf6-bbb7-091e655a1a7c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:46.069451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:46.153290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec81200-c71d-4cf6-bbb7-091e655a1a7c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:46.189133 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312866c-86e4-4234-8213-5391f3e3ac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:46.189773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312866c-86e4-4234-8213-5391f3e3ac40 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:46.190050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312866c-86e4-4234-8213-5391f3e3ac40 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:46.190326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312866c-86e4-4234-8213-5391f3e3ac40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:46.202246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312866c-86e4-4234-8213-5391f3e3ac40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e312866c-86e4-4234-8213-5391f3e3ac40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:46.207080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:46.207988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:46.210308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 146/289] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:46.214790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4780d-6eb1-40f3-9d6f-a633af4c5d83 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:46.266116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:46.344657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4780d-6eb1-40f3-9d6f-a633af4c5d83 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:46.402376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdbe84-c5a4-4411-83ab-dc1da2c29b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:46.403143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdbe84-c5a4-4411-83ab-dc1da2c29b53 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:46.403299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdbe84-c5a4-4411-83ab-dc1da2c29b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:46.403549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdbe84-c5a4-4411-83ab-dc1da2c29b53 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:46.404242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdbe84-c5a4-4411-83ab-dc1da2c29b53 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:46.404542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdbe84-c5a4-4411-83ab-dc1da2c29b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:46.405902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdbe84-c5a4-4411-83ab-dc1da2c29b53 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:46.415559 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 144/290] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:46.424856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312866c-86e4-4234-8213-5391f3e3ac40 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:46.547522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:46.565368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312866c-86e4-4234-8213-5391f3e3ac40 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:46.603089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1b8d-8158-460a-8618-69f48bd3dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:46.603650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1b8d-8158-460a-8618-69f48bd3dd71 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:46.603961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1b8d-8158-460a-8618-69f48bd3dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:46.604239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db1b8d-8158-460a-8618-69f48bd3dd71 None admin] PATH_INFO: `/v3/users/15fe769a5ea24b4d8bb81a2353520127` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:46.605494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db1b8d-8158-460a-8618-69f48bd3dd71 None admin] RBAC: Authorizing `identity:get_user(user_id=15fe769a5ea24b4d8bb81a2353520127)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:46.606952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db1b8d-8158-460a-8618-69f48bd3dd71 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:46.608951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 147/291] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:46 2022] GET /identity/v3/users/15fe769a5ea24b4d8bb81a2353520127 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:46.617163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdbe84-c5a4-4411-83ab-dc1da2c29b53 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:46.790177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdbe84-c5a4-4411-83ab-dc1da2c29b53 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:46.816145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:46.827775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26f439-c13e-47dc-9668-de3434393244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:46.828396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26f439-c13e-47dc-9668-de3434393244 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:46.829024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26f439-c13e-47dc-9668-de3434393244 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:46.829734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26f439-c13e-47dc-9668-de3434393244 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:46.902094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26f439-c13e-47dc-9668-de3434393244 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:46.903548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26f439-c13e-47dc-9668-de3434393244 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:47.044752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 145/292] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:47.050875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1b8d-8158-460a-8618-69f48bd3dd71 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:47.191993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db1b8d-8158-460a-8618-69f48bd3dd71 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:47.228108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac1c3af-b733-46f9-bd15-c6c1b694042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:47.228741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1c3af-b733-46f9-bd15-c6c1b694042c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:47.229020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1c3af-b733-46f9-bd15-c6c1b694042c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:47.229290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac1c3af-b733-46f9-bd15-c6c1b694042c None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:47.231163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1c3af-b733-46f9-bd15-c6c1b694042c None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:47.232967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac1c3af-b733-46f9-bd15-c6c1b694042c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:47.234950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 148/293] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:47 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:47.240987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26f439-c13e-47dc-9668-de3434393244 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:47.254656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:47.437644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26f439-c13e-47dc-9668-de3434393244 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:47.529151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc13ef1-01ca-4ccd-a7dc-9c9954723c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:47.529663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc13ef1-01ca-4ccd-a7dc-9c9954723c7e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:47.529945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc13ef1-01ca-4ccd-a7dc-9c9954723c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:47.530214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc13ef1-01ca-4ccd-a7dc-9c9954723c7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:47.531001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc13ef1-01ca-4ccd-a7dc-9c9954723c7e None admin] RBAC: Adding query filter params (role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=a06126b4287647ca9294ce703dd39691, user.id=15fe769a5ea24b4d8bb81a2353520127) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:47.531316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc13ef1-01ca-4ccd-a7dc-9c9954723c7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:47.532787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc13ef1-01ca-4ccd-a7dc-9c9954723c7e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:47.537000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b138fb7-d003-4dc0-b1ea-1c7af3c8f66f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:47.537743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b138fb7-d003-4dc0-b1ea-1c7af3c8f66f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:47.537743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b138fb7-d003-4dc0-b1ea-1c7af3c8f66f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:47.539166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 149/294] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:47.539575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 146/295] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:47 2022] GET /identity/v3/role_assignments?user.id=15fe769a5ea24b4d8bb81a2353520127&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 608 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:47.562237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd10ca-fc99-43b3-8ca5-3816f3ec6e89 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:47.562529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd10ca-fc99-43b3-8ca5-3816f3ec6e89 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:47.562832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd10ca-fc99-43b3-8ca5-3816f3ec6e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:47.596996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9cfd10ca-fc99-43b3-8ca5-3816f3ec6e89 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:47.598241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9cfd10ca-fc99-43b3-8ca5-3816f3ec6e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:47.627232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:47.668741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3099f6-35b1-441a-86fd-a57ecc9eb3ba None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:47.669093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3099f6-35b1-441a-86fd-a57ecc9eb3ba None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:47.669859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3099f6-35b1-441a-86fd-a57ecc9eb3ba None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:47.671686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 147/296] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:47.678816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788be5c2-996f-4c84-afd5-51ade60bec5d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:47.679119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788be5c2-996f-4c84-afd5-51ade60bec5d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:47.679367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788be5c2-996f-4c84-afd5-51ade60bec5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:47.712952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-788be5c2-996f-4c84-afd5-51ade60bec5d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:47.719792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-788be5c2-996f-4c84-afd5-51ade60bec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:47.734843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 150/297] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:47.803598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:47.810074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 148/298] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:47.947956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177c758-031d-4a44-9f74-13052be922d8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:47.948833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177c758-031d-4a44-9f74-13052be922d8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:47.949069 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177c758-031d-4a44-9f74-13052be922d8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:47.950669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 151/299] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:47.957616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f5d25-f209-4899-a3ed-5fecad0386cb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:47.958173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f5d25-f209-4899-a3ed-5fecad0386cb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:47.958649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f5d25-f209-4899-a3ed-5fecad0386cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:47.983918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-516f5d25-f209-4899-a3ed-5fecad0386cb None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:47.985452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-516f5d25-f209-4899-a3ed-5fecad0386cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:48.033199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177c758-031d-4a44-9f74-13052be922d8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:48.071787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 149/300] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:48.145739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f5d25-f209-4899-a3ed-5fecad0386cb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:48.166175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177c758-031d-4a44-9f74-13052be922d8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:48.204848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996b521-d881-4e36-a3a5-8e50f3fd7f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:48.205533 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996b521-d881-4e36-a3a5-8e50f3fd7f1d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.205812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996b521-d881-4e36-a3a5-8e50f3fd7f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.206083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996b521-d881-4e36-a3a5-8e50f3fd7f1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.207304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996b521-d881-4e36-a3a5-8e50f3fd7f1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:48.208787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996b521-d881-4e36-a3a5-8e50f3fd7f1d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:48.210919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 152/301] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:48.217177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996b521-d881-4e36-a3a5-8e50f3fd7f1d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:48.242131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:48.260404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f5d25-f209-4899-a3ed-5fecad0386cb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:48.293445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:48.300062 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5d98e-cabf-4ce7-80c9-bac47722004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:48.300791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5d98e-cabf-4ce7-80c9-bac47722004d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.301148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5d98e-cabf-4ce7-80c9-bac47722004d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.301520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5d98e-cabf-4ce7-80c9-bac47722004d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.314618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5d98e-cabf-4ce7-80c9-bac47722004d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-37b5d98e-cabf-4ce7-80c9-bac47722004d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:48.320068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:48.321130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:48.322157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:48.323840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 150/302] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:06:48.327082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5d98e-cabf-4ce7-80c9-bac47722004d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:48.341161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996b521-d881-4e36-a3a5-8e50f3fd7f1d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:48.383782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adcfad-2baf-4f5f-91a1-f3b49369b032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:48.384408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adcfad-2baf-4f5f-91a1-f3b49369b032 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.384714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adcfad-2baf-4f5f-91a1-f3b49369b032 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.384940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adcfad-2baf-4f5f-91a1-f3b49369b032 None admin] PATH_INFO: `/v3/groups` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.385634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adcfad-2baf-4f5f-91a1-f3b49369b032 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:48.387106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adcfad-2baf-4f5f-91a1-f3b49369b032 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:48.407985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 153/303] 10.222.0.71 () {60 vars in 1134 bytes} [Wed Sep 21 12:06:48 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:06:48.410043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adcfad-2baf-4f5f-91a1-f3b49369b032 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:48.465183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5d98e-cabf-4ce7-80c9-bac47722004d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:48.510245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4a8cc-5c39-43d7-9727-e32c6fa712e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:48.511205 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4a8cc-5c39-43d7-9727-e32c6fa712e9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.511634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4a8cc-5c39-43d7-9727-e32c6fa712e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.512079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4a8cc-5c39-43d7-9727-e32c6fa712e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.512968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4a8cc-5c39-43d7-9727-e32c6fa712e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:48.513592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4a8cc-5c39-43d7-9727-e32c6fa712e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:48.516046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4a8cc-5c39-43d7-9727-e32c6fa712e9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:48.526765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 151/304] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:48.532768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4a8cc-5c39-43d7-9727-e32c6fa712e9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:48.546093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:48.571515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adcfad-2baf-4f5f-91a1-f3b49369b032 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:48.601657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:48.615871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e4367-54e8-4616-b4c8-735f77701cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:48.616837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e4367-54e8-4616-b4c8-735f77701cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.617266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e4367-54e8-4616-b4c8-735f77701cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.617687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e4367-54e8-4616-b4c8-735f77701cd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.632372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e4367-54e8-4616-b4c8-735f77701cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-434e4367-54e8-4616-b4c8-735f77701cd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:48.638871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:48.640111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:48.640933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:48.643565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 154/305] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:48.650018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e4367-54e8-4616-b4c8-735f77701cd2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:48.679691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4a8cc-5c39-43d7-9727-e32c6fa712e9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:48.718463 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0637383-4538-455b-9c0d-7e540afb7e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:48.719653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0637383-4538-455b-9c0d-7e540afb7e2c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.720061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0637383-4538-455b-9c0d-7e540afb7e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.720399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0637383-4538-455b-9c0d-7e540afb7e2c None admin] PATH_INFO: `/v3/users/b262eee2b1d946d388511b5d27a9f5fc` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.735564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0637383-4538-455b-9c0d-7e540afb7e2c None admin] RBAC: Authorizing `identity:get_user(user_id=b262eee2b1d946d388511b5d27a9f5fc)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:48.737156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0637383-4538-455b-9c0d-7e540afb7e2c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:48.743875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 152/306] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:48 2022] GET /identity/v3/users/b262eee2b1d946d388511b5d27a9f5fc => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:48.745789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734831a-1a25-4309-9cfe-392a2d32a7d5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.746075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734831a-1a25-4309-9cfe-392a2d32a7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.746345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734831a-1a25-4309-9cfe-392a2d32a7d5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.747651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 153/307] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:48.748843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734831a-1a25-4309-9cfe-392a2d32a7d5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:48.778322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e4367-54e8-4616-b4c8-735f77701cd2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:48.813382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14d483-53dd-404f-abaa-f337a3f919a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:48.813993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14d483-53dd-404f-abaa-f337a3f919a8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.814267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14d483-53dd-404f-abaa-f337a3f919a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.814535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14d483-53dd-404f-abaa-f337a3f919a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.815214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14d483-53dd-404f-abaa-f337a3f919a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:48.815512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14d483-53dd-404f-abaa-f337a3f919a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:48.816936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14d483-53dd-404f-abaa-f337a3f919a8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:48.830243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 155/308] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:48.830243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552a638-2eec-4f5d-8c1a-00199ba69ea1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.830243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552a638-2eec-4f5d-8c1a-00199ba69ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.830243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552a638-2eec-4f5d-8c1a-00199ba69ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.834252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:48.849243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6552a638-2eec-4f5d-8c1a-00199ba69ea1 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:48.850419 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6552a638-2eec-4f5d-8c1a-00199ba69ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:48.875417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734831a-1a25-4309-9cfe-392a2d32a7d5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:48.894937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:48.921854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c028052-31f6-4264-bb7e-19f823baec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:48.922351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 156/309] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:48.922454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c028052-31f6-4264-bb7e-19f823baec1d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:48.922720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c028052-31f6-4264-bb7e-19f823baec1d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:48.922990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c028052-31f6-4264-bb7e-19f823baec1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:48.923713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c028052-31f6-4264-bb7e-19f823baec1d None admin] RBAC: Adding query filter params (role.id=9dbcf5e8a10a4728888b9f4e923f5814, user.id=b262eee2b1d946d388511b5d27a9f5fc, scope.system=all) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:48.924047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c028052-31f6-4264-bb7e-19f823baec1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:48.924330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552a638-2eec-4f5d-8c1a-00199ba69ea1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:48.925471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c028052-31f6-4264-bb7e-19f823baec1d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:48.931011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 154/310] 10.222.0.71 () {56 vars in 1308 bytes} [Wed Sep 21 12:06:48 2022] GET /identity/v3/role_assignments?user.id=b262eee2b1d946d388511b5d27a9f5fc&scope.system=all&role.id=9dbcf5e8a10a4728888b9f4e923f5814 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:49.041640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552a638-2eec-4f5d-8c1a-00199ba69ea1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:49.080205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:49.082149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb2cf-6609-428f-ae0b-d190b0221135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:49.082805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb2cf-6609-428f-ae0b-d190b0221135 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:49.083136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb2cf-6609-428f-ae0b-d190b0221135 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:49.083414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ceb2cf-6609-428f-ae0b-d190b0221135 None admin] PATH_INFO: `/v3/users/362ffed0b1804934b37ac5c4a93bc457` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:49.084854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceb2cf-6609-428f-ae0b-d190b0221135 None admin] RBAC: Authorizing `identity:get_user(user_id=362ffed0b1804934b37ac5c4a93bc457)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:49.086436 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ceb2cf-6609-428f-ae0b-d190b0221135 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:49.093966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 157/311] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:48 2022] GET /identity/v3/users/362ffed0b1804934b37ac5c4a93bc457 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:49.094996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c028052-31f6-4264-bb7e-19f823baec1d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:49.200117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c028052-31f6-4264-bb7e-19f823baec1d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:49.216072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb2cf-6609-428f-ae0b-d190b0221135 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:49.249042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82676043-1f06-4a72-9110-0f3e670a30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:49.250267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82676043-1f06-4a72-9110-0f3e670a30ec None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:49.251222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82676043-1f06-4a72-9110-0f3e670a30ec None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:49.251978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82676043-1f06-4a72-9110-0f3e670a30ec None admin] PATH_INFO: `/v3/system/users/362ffed0b1804934b37ac5c4a93bc457/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:49.254265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82676043-1f06-4a72-9110-0f3e670a30ec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=362ffed0b1804934b37ac5c4a93bc457, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:49.256191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82676043-1f06-4a72-9110-0f3e670a30ec None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:49.275795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 155/312] 10.222.0.71 () {58 vars in 1334 bytes} [Wed Sep 21 12:06:49 2022] PUT /identity/v3/system/users/362ffed0b1804934b37ac5c4a93bc457/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:49.357372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ceb2cf-6609-428f-ae0b-d190b0221135 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:49.402290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7827273-1356-41eb-808a-5a4a3bd4e149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:49.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7827273-1356-41eb-808a-5a4a3bd4e149 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:49.403913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7827273-1356-41eb-808a-5a4a3bd4e149 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:49.404352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7827273-1356-41eb-808a-5a4a3bd4e149 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:49.417578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7827273-1356-41eb-808a-5a4a3bd4e149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a7827273-1356-41eb-808a-5a4a3bd4e149 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:49.422991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:49.423962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:49.424769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:49.426863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 158/313] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:06:49.433846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82676043-1f06-4a72-9110-0f3e670a30ec None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:49.483091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:49.549586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82676043-1f06-4a72-9110-0f3e670a30ec None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:49.587568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481e56d-e6a9-496e-8708-36e9e167ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:49.588470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481e56d-e6a9-496e-8708-36e9e167ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:49.589053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481e56d-e6a9-496e-8708-36e9e167ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:49.589546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481e56d-e6a9-496e-8708-36e9e167ddf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:49.590444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481e56d-e6a9-496e-8708-36e9e167ddf4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:49.591132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481e56d-e6a9-496e-8708-36e9e167ddf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:49.592732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481e56d-e6a9-496e-8708-36e9e167ddf4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:49.601634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 156/314] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:49.607598 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7827273-1356-41eb-808a-5a4a3bd4e149 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:49.675261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:49.720705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7827273-1356-41eb-808a-5a4a3bd4e149 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:49.757008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636227-0177-499f-81e1-2e76aa42511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:49.757970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636227-0177-499f-81e1-2e76aa42511b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:49.758425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636227-0177-499f-81e1-2e76aa42511b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:49.758827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df636227-0177-499f-81e1-2e76aa42511b None admin] PATH_INFO: `/v3/users/e2d6fab82d444951bb8dd691f084f29c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:49.760258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df636227-0177-499f-81e1-2e76aa42511b None admin] RBAC: Authorizing `identity:get_user(user_id=e2d6fab82d444951bb8dd691f084f29c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:49.762639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df636227-0177-499f-81e1-2e76aa42511b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:49.765049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 159/315] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:49 2022] GET /identity/v3/users/e2d6fab82d444951bb8dd691f084f29c => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:49.770739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481e56d-e6a9-496e-8708-36e9e167ddf4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:49.879502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:49.883369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481e56d-e6a9-496e-8708-36e9e167ddf4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:49.922157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0025b5-9557-4036-a689-19754674c8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:49.922999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0025b5-9557-4036-a689-19754674c8fb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:49.923416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0025b5-9557-4036-a689-19754674c8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:49.923878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0025b5-9557-4036-a689-19754674c8fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:49.992237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0025b5-9557-4036-a689-19754674c8fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:49.993709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0025b5-9557-4036-a689-19754674c8fb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:50.088494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:50.092339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 157/316] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:50.097944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636227-0177-499f-81e1-2e76aa42511b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:50.207940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df636227-0177-499f-81e1-2e76aa42511b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:50.236934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:50.245229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1ecad7-4bbc-4a7c-9fcf-a852a4cc7d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:50.246312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ecad7-4bbc-4a7c-9fcf-a852a4cc7d58 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:50.246950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ecad7-4bbc-4a7c-9fcf-a852a4cc7d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:50.247457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1ecad7-4bbc-4a7c-9fcf-a852a4cc7d58 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:50.248992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ecad7-4bbc-4a7c-9fcf-a852a4cc7d58 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:50.250508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1ecad7-4bbc-4a7c-9fcf-a852a4cc7d58 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:50.252502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 160/317] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:50 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:50.258764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0025b5-9557-4036-a689-19754674c8fb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:50.366562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0025b5-9557-4036-a689-19754674c8fb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:50.401355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2d7e1-0c39-47cc-8e13-2599453e84ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:50.401957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2d7e1-0c39-47cc-8e13-2599453e84ba None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:50.402235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2d7e1-0c39-47cc-8e13-2599453e84ba None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:50.402504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2d7e1-0c39-47cc-8e13-2599453e84ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:50.403218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2d7e1-0c39-47cc-8e13-2599453e84ba None admin] RBAC: Adding query filter params (role.id=9dbcf5e8a10a4728888b9f4e923f5814, scope.project.id=a06126b4287647ca9294ce703dd39691, user.id=e2d6fab82d444951bb8dd691f084f29c) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:50.403525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2d7e1-0c39-47cc-8e13-2599453e84ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:50.404981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2d7e1-0c39-47cc-8e13-2599453e84ba None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:50.411045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 158/318] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:06:50 2022] GET /identity/v3/role_assignments?user.id=e2d6fab82d444951bb8dd691f084f29c&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=9dbcf5e8a10a4728888b9f4e923f5814 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:50.751819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8e18a-b1c3-4b8f-9487-bbed94dd421d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:50.752391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8e18a-b1c3-4b8f-9487-bbed94dd421d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:50.752391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8e18a-b1c3-4b8f-9487-bbed94dd421d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:50.753796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 161/319] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:50.760361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97123a-6869-4d6d-b2d2-44731f59310e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:50.760361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97123a-6869-4d6d-b2d2-44731f59310e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:50.760361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b97123a-6869-4d6d-b2d2-44731f59310e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:50.778051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3b97123a-6869-4d6d-b2d2-44731f59310e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:50.779226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3b97123a-6869-4d6d-b2d2-44731f59310e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:50.843155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:50.852249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 159/320] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:51.121060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8e18a-b1c3-4b8f-9487-bbed94dd421d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:51.183360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b62b4-d653-4655-8c61-2d11d602cb40 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:51.183647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b62b4-d653-4655-8c61-2d11d602cb40 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:51.183946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b62b4-d653-4655-8c61-2d11d602cb40 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:51.185466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 160/321] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:51.190359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5b882-38a2-409e-a681-53687da52cbb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:51.190644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5b882-38a2-409e-a681-53687da52cbb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:51.190922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5b882-38a2-409e-a681-53687da52cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:51.209733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cac5b882-38a2-409e-a681-53687da52cbb None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:51.210924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cac5b882-38a2-409e-a681-53687da52cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:51.228921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8e18a-b1c3-4b8f-9487-bbed94dd421d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:51.265865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e6c09-4282-49f7-bcd7-7a2325cfce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:51.266499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e6c09-4282-49f7-bcd7-7a2325cfce2b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:51.266780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e6c09-4282-49f7-bcd7-7a2325cfce2b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:51.267049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e6c09-4282-49f7-bcd7-7a2325cfce2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:51.268595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e6c09-4282-49f7-bcd7-7a2325cfce2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:51.270577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e6c09-4282-49f7-bcd7-7a2325cfce2b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:51.277708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 162/322] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:06:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:51.278982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e6c09-4282-49f7-bcd7-7a2325cfce2b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:51.281610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 161/323] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:51.345763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:51.385493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e6c09-4282-49f7-bcd7-7a2325cfce2b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:51.419998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31cdc02-b52a-48fc-9026-8f736b004e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:51.420606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cdc02-b52a-48fc-9026-8f736b004e73 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:51.420942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cdc02-b52a-48fc-9026-8f736b004e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:51.421219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31cdc02-b52a-48fc-9026-8f736b004e73 None admin] PATH_INFO: `/v3/groups` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:51.421848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cdc02-b52a-48fc-9026-8f736b004e73 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:51.423342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31cdc02-b52a-48fc-9026-8f736b004e73 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:51.444055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 163/324] 10.222.0.71 () {60 vars in 1134 bytes} [Wed Sep 21 12:06:51 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:06:51.552013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5b882-38a2-409e-a681-53687da52cbb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:51.602958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36fa9f-edac-402d-8688-f2e65b682edd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:51.603247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36fa9f-edac-402d-8688-f2e65b682edd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:51.603526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36fa9f-edac-402d-8688-f2e65b682edd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:51.605144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 164/325] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:06:51.610915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37655d-6c1e-4cd7-a93a-5434be10ca2a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:51.611269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37655d-6c1e-4cd7-a93a-5434be10ca2a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:51.611647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37655d-6c1e-4cd7-a93a-5434be10ca2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:51.629859 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0a37655d-6c1e-4cd7-a93a-5434be10ca2a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:51.631068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0a37655d-6c1e-4cd7-a93a-5434be10ca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:51.671703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5b882-38a2-409e-a681-53687da52cbb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:51.702828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:51.708883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 165/326] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:06:51.719332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5559073-f854-4819-bf1b-95225995371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:51.720151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5559073-f854-4819-bf1b-95225995371e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:51.720531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5559073-f854-4819-bf1b-95225995371e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:51.720814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5559073-f854-4819-bf1b-95225995371e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:51.734271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5559073-f854-4819-bf1b-95225995371e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d5559073-f854-4819-bf1b-95225995371e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:51.739294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:51.740404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:51.741393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 162/327] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:51.746448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37655d-6c1e-4cd7-a93a-5434be10ca2a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:51.812405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:51.866569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37655d-6c1e-4cd7-a93a-5434be10ca2a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:51.907509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227bb92-95d3-4976-9884-dbfff94636c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:51.908171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227bb92-95d3-4976-9884-dbfff94636c5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:51.908475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227bb92-95d3-4976-9884-dbfff94636c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:51.909005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227bb92-95d3-4976-9884-dbfff94636c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:51.909687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227bb92-95d3-4976-9884-dbfff94636c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:51.909992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227bb92-95d3-4976-9884-dbfff94636c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:51.911439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227bb92-95d3-4976-9884-dbfff94636c5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:51.920764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 166/328] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:51.926024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5559073-f854-4819-bf1b-95225995371e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:51.992003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227bb92-95d3-4976-9884-dbfff94636c5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:52.039788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5559073-f854-4819-bf1b-95225995371e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:52.077087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed8f42-0643-4136-8973-fc91f6161a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:52.077884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed8f42-0643-4136-8973-fc91f6161a2b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:52.078264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed8f42-0643-4136-8973-fc91f6161a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:52.078654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed8f42-0643-4136-8973-fc91f6161a2b None admin] PATH_INFO: `/v3/users/b262eee2b1d946d388511b5d27a9f5fc` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:52.080000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed8f42-0643-4136-8973-fc91f6161a2b None admin] RBAC: Authorizing `identity:get_user(user_id=b262eee2b1d946d388511b5d27a9f5fc)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:52.081656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed8f42-0643-4136-8973-fc91f6161a2b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:52.083811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 163/329] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:51 2022] GET /identity/v3/users/b262eee2b1d946d388511b5d27a9f5fc => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:52.089389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed8f42-0643-4136-8973-fc91f6161a2b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:52.106787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227bb92-95d3-4976-9884-dbfff94636c5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:52.124973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:52.154270 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1da4b-c9b6-4250-829f-44cdc7898e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:52.154911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1da4b-c9b6-4250-829f-44cdc7898e57 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:52.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1da4b-c9b6-4250-829f-44cdc7898e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:52.155460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1da4b-c9b6-4250-829f-44cdc7898e57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:52.157854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:52.169059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1da4b-c9b6-4250-829f-44cdc7898e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e4a1da4b-c9b6-4250-829f-44cdc7898e57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:52.174947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:52.175835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:52.176653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:52.179292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 167/330] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:06:52.185316 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1da4b-c9b6-4250-829f-44cdc7898e57 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:52.216759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed8f42-0643-4136-8973-fc91f6161a2b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:52.259862 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f00ee-a472-44b3-b873-314f37691973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:52.261153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f00ee-a472-44b3-b873-314f37691973 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:52.261360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f00ee-a472-44b3-b873-314f37691973 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:52.261638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f00ee-a472-44b3-b873-314f37691973 None admin] PATH_INFO: `/v3/system/users/b262eee2b1d946d388511b5d27a9f5fc/roles/9dbcf5e8a10a4728888b9f4e923f5814` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:52.263078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f00ee-a472-44b3-b873-314f37691973 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b262eee2b1d946d388511b5d27a9f5fc, role_id=9dbcf5e8a10a4728888b9f4e923f5814)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:52.264787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f00ee-a472-44b3-b873-314f37691973 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:52.273070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 164/331] 10.222.0.71 () {58 vars in 1334 bytes} [Wed Sep 21 12:06:52 2022] PUT /identity/v3/system/users/b262eee2b1d946d388511b5d27a9f5fc/roles/9dbcf5e8a10a4728888b9f4e923f5814 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:06:52.311423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1da4b-c9b6-4250-829f-44cdc7898e57 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:52.355901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001ca51-771d-4667-ac93-ea5a37c15e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:52.357027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001ca51-771d-4667-ac93-ea5a37c15e15 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:52.357313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001ca51-771d-4667-ac93-ea5a37c15e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:52.357732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001ca51-771d-4667-ac93-ea5a37c15e15 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:52.358759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001ca51-771d-4667-ac93-ea5a37c15e15 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:52.359116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001ca51-771d-4667-ac93-ea5a37c15e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:52.361027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001ca51-771d-4667-ac93-ea5a37c15e15 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:52.372641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 168/332] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:52.381007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f00ee-a472-44b3-b873-314f37691973 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:52.534286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f00ee-a472-44b3-b873-314f37691973 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:52.556696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:52.593849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccf38a8-d92b-49b9-9fea-59c8cae2198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:52.595186 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf38a8-d92b-49b9-9fea-59c8cae2198f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:52.595875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf38a8-d92b-49b9-9fea-59c8cae2198f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:52.596600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccf38a8-d92b-49b9-9fea-59c8cae2198f None admin] PATH_INFO: `/v3/users/362ffed0b1804934b37ac5c4a93bc457` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:52.599103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf38a8-d92b-49b9-9fea-59c8cae2198f None admin] RBAC: Authorizing `identity:get_user(user_id=362ffed0b1804934b37ac5c4a93bc457)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:52.601994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccf38a8-d92b-49b9-9fea-59c8cae2198f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:52.606323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 165/333] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:52 2022] GET /identity/v3/users/362ffed0b1804934b37ac5c4a93bc457 => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:52.613351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001ca51-771d-4667-ac93-ea5a37c15e15 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:52.658965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:52.726564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001ca51-771d-4667-ac93-ea5a37c15e15 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:52.773877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ad45f-c745-4961-9c58-a61f34a66b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:52.774511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ad45f-c745-4961-9c58-a61f34a66b0f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:52.774790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ad45f-c745-4961-9c58-a61f34a66b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:52.775059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ad45f-c745-4961-9c58-a61f34a66b0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:52.775828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ad45f-c745-4961-9c58-a61f34a66b0f None admin] RBAC: Adding query filter params (role.id=3367be0edc604df081b5ac7a9e2a7b92, user.id=362ffed0b1804934b37ac5c4a93bc457, scope.system=all) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:52.776182 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ad45f-c745-4961-9c58-a61f34a66b0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:52.777679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ad45f-c745-4961-9c58-a61f34a66b0f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:52.796335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 169/334] 10.222.0.71 () {56 vars in 1308 bytes} [Wed Sep 21 12:06:52 2022] GET /identity/v3/role_assignments?user.id=362ffed0b1804934b37ac5c4a93bc457&scope.system=all&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:54.389845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3287ae-98e0-4e5a-9327-2b9532f5225f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:54.390870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3287ae-98e0-4e5a-9327-2b9532f5225f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:54.391369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3287ae-98e0-4e5a-9327-2b9532f5225f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:54.393335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 166/335] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:54.399486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99005-7d05-46c6-9d46-b151e64123ff None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:54.400002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99005-7d05-46c6-9d46-b151e64123ff None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:54.400084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99005-7d05-46c6-9d46-b151e64123ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:54.425423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-44b99005-7d05-46c6-9d46-b151e64123ff None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:54.426639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44b99005-7d05-46c6-9d46-b151e64123ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:54.514310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 170/336] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:54.521680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa2f9c-cff4-4953-8be9-80db1fa591d0 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:54.521961 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa2f9c-cff4-4953-8be9-80db1fa591d0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:54.522231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa2f9c-cff4-4953-8be9-80db1fa591d0 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:54.523567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 167/337] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:54.529772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb06771-71fe-495d-8799-afbbed2c230e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:54.530067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb06771-71fe-495d-8799-afbbed2c230e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:54.530345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb06771-71fe-495d-8799-afbbed2c230e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:54.550181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-deb06771-71fe-495d-8799-afbbed2c230e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:54.551365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-deb06771-71fe-495d-8799-afbbed2c230e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:54.579037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:54.645812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 171/338] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:55.119915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa2f9c-cff4-4953-8be9-80db1fa591d0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:55.228864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb06771-71fe-495d-8799-afbbed2c230e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:55.253694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa2f9c-cff4-4953-8be9-80db1fa591d0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:55.309278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bc021-9c65-47e3-b337-68e448156dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:55.309921 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bc021-9c65-47e3-b337-68e448156dfe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:55.310193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bc021-9c65-47e3-b337-68e448156dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:55.310457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bc021-9c65-47e3-b337-68e448156dfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:55.335936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bc021-9c65-47e3-b337-68e448156dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-752bc021-9c65-47e3-b337-68e448156dfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:55.341805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:55.342649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:55.343415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:55.349858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 168/339] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:55.357360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bc021-9c65-47e3-b337-68e448156dfe None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:55.451804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:55.471792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb06771-71fe-495d-8799-afbbed2c230e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:55.519087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bc021-9c65-47e3-b337-68e448156dfe None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:55.523539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe70b8-baa9-4f6e-999d-bdd6f400a525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:55.524268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe70b8-baa9-4f6e-999d-bdd6f400a525 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:55.524552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe70b8-baa9-4f6e-999d-bdd6f400a525 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:55.524825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe70b8-baa9-4f6e-999d-bdd6f400a525 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:55.540379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe70b8-baa9-4f6e-999d-bdd6f400a525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6dbe70b8-baa9-4f6e-999d-bdd6f400a525 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:55.545742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:55.546675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:55.547475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 172/340] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:06:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:55.548709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56bf2-cf10-4c40-846a-423362ad3a6e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:55.548709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56bf2-cf10-4c40-846a-423362ad3a6e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:55.549041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d56bf2-cf10-4c40-846a-423362ad3a6e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:55.550253 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 173/341] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:55.551281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298243a-0f14-4e9c-af67-49af86361653 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:55.551561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298243a-0f14-4e9c-af67-49af86361653 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:55.552510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298243a-0f14-4e9c-af67-49af86361653 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:55.554293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 174/342] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:55.563153 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055f89e-b548-41b0-9ea4-91f0b1c2c034 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:55.563423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055f89e-b548-41b0-9ea4-91f0b1c2c034 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:55.563651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c05a4-b2ac-48a5-b69e-c79956a57fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:55.563877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055f89e-b548-41b0-9ea4-91f0b1c2c034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:55.564368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c05a4-b2ac-48a5-b69e-c79956a57fdd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:55.564822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c05a4-b2ac-48a5-b69e-c79956a57fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:55.565102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c05a4-b2ac-48a5-b69e-c79956a57fdd None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:55.565942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c05a4-b2ac-48a5-b69e-c79956a57fdd None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:55.566288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c05a4-b2ac-48a5-b69e-c79956a57fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:55.567696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c05a4-b2ac-48a5-b69e-c79956a57fdd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:55.581420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 169/343] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:55.583054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1607e-4dcb-4af0-8237-664f5c3d447c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:55.583563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1607e-4dcb-4af0-8237-664f5c3d447c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:55.584076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1607e-4dcb-4af0-8237-664f5c3d447c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:55.604528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cff1607e-4dcb-4af0-8237-664f5c3d447c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:55.604948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4055f89e-b548-41b0-9ea4-91f0b1c2c034 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:55.606133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cff1607e-4dcb-4af0-8237-664f5c3d447c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:55.606325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4055f89e-b548-41b0-9ea4-91f0b1c2c034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:55.618287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:55.686408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 170/344] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:55.687525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1607e-4dcb-4af0-8237-664f5c3d447c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:55.692904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 175/345] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:55.693869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055f89e-b548-41b0-9ea4-91f0b1c2c034 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:55.756850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:55.876839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1607e-4dcb-4af0-8237-664f5c3d447c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:55.889033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055f89e-b548-41b0-9ea4-91f0b1c2c034 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:55.943895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1fad8-e533-4d8c-8faa-c219a2257ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:55.944517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1fad8-e533-4d8c-8faa-c219a2257ebf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:55.944790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1fad8-e533-4d8c-8faa-c219a2257ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:55.945126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1fad8-e533-4d8c-8faa-c219a2257ebf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:55.963498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:55.964994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01577e25-855a-4318-8ea6-46ece0d064a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:55.965640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01577e25-855a-4318-8ea6-46ece0d064a4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:55.966174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01577e25-855a-4318-8ea6-46ece0d064a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:55.969034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01577e25-855a-4318-8ea6-46ece0d064a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:55.969034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01577e25-855a-4318-8ea6-46ece0d064a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:55.969034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01577e25-855a-4318-8ea6-46ece0d064a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:55.969034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01577e25-855a-4318-8ea6-46ece0d064a4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:55.985923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 171/346] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:06:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:55.994530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01577e25-855a-4318-8ea6-46ece0d064a4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:56.044045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1fad8-e533-4d8c-8faa-c219a2257ebf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:56.045915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1fad8-e533-4d8c-8faa-c219a2257ebf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:56.198459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 176/347] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:56.204429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1fad8-e533-4d8c-8faa-c219a2257ebf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:56.212753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:56.218461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01577e25-855a-4318-8ea6-46ece0d064a4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:56.262639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ef3b3-2663-4ae9-8ccb-85d304bb42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:56.263989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ef3b3-2663-4ae9-8ccb-85d304bb42ff None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:56.263989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ef3b3-2663-4ae9-8ccb-85d304bb42ff None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:56.263989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ef3b3-2663-4ae9-8ccb-85d304bb42ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:56.284714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:56.351402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ef3b3-2663-4ae9-8ccb-85d304bb42ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:56.353995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ef3b3-2663-4ae9-8ccb-85d304bb42ff None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:56.359845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1fad8-e533-4d8c-8faa-c219a2257ebf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:56.404426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717883e-52f5-446d-a87d-31186ac0b2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:56.405111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717883e-52f5-446d-a87d-31186ac0b2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:56.405389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717883e-52f5-446d-a87d-31186ac0b2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:56.405663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717883e-52f5-446d-a87d-31186ac0b2e8 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:56.406853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717883e-52f5-446d-a87d-31186ac0b2e8 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:56.408880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b717883e-52f5-446d-a87d-31186ac0b2e8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:56.410245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 177/348] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:56 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:56.410951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717883e-52f5-446d-a87d-31186ac0b2e8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:56.493805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:56.501713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 172/349] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 12:06:56.502651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ef3b3-2663-4ae9-8ccb-85d304bb42ff None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:56.546953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717883e-52f5-446d-a87d-31186ac0b2e8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:56.562520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:56.589954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e39498-d448-4abe-88e0-95f0a542bbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:56.590606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e39498-d448-4abe-88e0-95f0a542bbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:56.591179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e39498-d448-4abe-88e0-95f0a542bbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:56.591452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e39498-d448-4abe-88e0-95f0a542bbf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:56.603015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e39498-d448-4abe-88e0-95f0a542bbf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e9e39498-d448-4abe-88e0-95f0a542bbf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:56.611171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:56.612183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 178/350] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:56.612980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670164c5-7ef1-4be9-91f4-defe3011821d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:56.613680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670164c5-7ef1-4be9-91f4-defe3011821d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:56.613680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670164c5-7ef1-4be9-91f4-defe3011821d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:56.619390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 179/351] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:56.619567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670164c5-7ef1-4be9-91f4-defe3011821d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:56.643206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ef3b3-2663-4ae9-8ccb-85d304bb42ff None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:56.705755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b97d8-8edb-4827-8976-cdd4a1990e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:56.707443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b97d8-8edb-4827-8976-cdd4a1990e5e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:56.707996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b97d8-8edb-4827-8976-cdd4a1990e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:56.708489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b97d8-8edb-4827-8976-cdd4a1990e5e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:56.722507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b97d8-8edb-4827-8976-cdd4a1990e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-709b97d8-8edb-4827-8976-cdd4a1990e5e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:56.727230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:56.728136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:56.728912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 173/352] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:56.730095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b97d8-8edb-4827-8976-cdd4a1990e5e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:56.774742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670164c5-7ef1-4be9-91f4-defe3011821d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:56.814204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:56.823437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ff51c-4ed8-41cd-a265-23fbb42cac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:56.824105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ff51c-4ed8-41cd-a265-23fbb42cac0c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:56.824383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ff51c-4ed8-41cd-a265-23fbb42cac0c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:56.824658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ff51c-4ed8-41cd-a265-23fbb42cac0c None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:56.833139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ff51c-4ed8-41cd-a265-23fbb42cac0c None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:56.835234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ff51c-4ed8-41cd-a265-23fbb42cac0c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:56.837167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 180/353] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:06:56 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:56.843542 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ff51c-4ed8-41cd-a265-23fbb42cac0c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:56.876865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b97d8-8edb-4827-8976-cdd4a1990e5e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:56.896958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:56.927746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4410ca7-2930-4a00-8f78-f4790820fa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:56.928503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4410ca7-2930-4a00-8f78-f4790820fa50 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:56.928775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4410ca7-2930-4a00-8f78-f4790820fa50 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:56.929041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4410ca7-2930-4a00-8f78-f4790820fa50 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:56.930232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4410ca7-2930-4a00-8f78-f4790820fa50 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:56.931712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4410ca7-2930-4a00-8f78-f4790820fa50 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:56.935302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 174/354] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:56 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:56.935638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea3171-a332-4296-bff4-67a988c1aae2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:56.935977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea3171-a332-4296-bff4-67a988c1aae2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:56.936251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea3171-a332-4296-bff4-67a988c1aae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:56.955118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-37ea3171-a332-4296-bff4-67a988c1aae2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:56.956345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-37ea3171-a332-4296-bff4-67a988c1aae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:56.971634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ff51c-4ed8-41cd-a265-23fbb42cac0c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:57.011384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de26fb-98ea-4809-b8df-904d5d69530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:57.012036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de26fb-98ea-4809-b8df-904d5d69530f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.012326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de26fb-98ea-4809-b8df-904d5d69530f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.012578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de26fb-98ea-4809-b8df-904d5d69530f None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.013250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de26fb-98ea-4809-b8df-904d5d69530f None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:57.013558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de26fb-98ea-4809-b8df-904d5d69530f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:57.014945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69de26fb-98ea-4809-b8df-904d5d69530f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:57.024669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 181/355] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:57.025306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de26fb-98ea-4809-b8df-904d5d69530f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:57.037869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 175/356] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:57.038794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea3171-a332-4296-bff4-67a988c1aae2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:57.108431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:57.154809 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:57.158118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de26fb-98ea-4809-b8df-904d5d69530f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:57.173508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea3171-a332-4296-bff4-67a988c1aae2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:57.198139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca610b0e-ad0e-478a-ba50-4c17cc94d435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:57.198763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610b0e-ad0e-478a-ba50-4c17cc94d435 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.199046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610b0e-ad0e-478a-ba50-4c17cc94d435 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.199451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610b0e-ad0e-478a-ba50-4c17cc94d435 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.200276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca610b0e-ad0e-478a-ba50-4c17cc94d435 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:57.200590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca610b0e-ad0e-478a-ba50-4c17cc94d435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:57.202010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca610b0e-ad0e-478a-ba50-4c17cc94d435 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:57.212251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 182/357] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:06:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:57.213401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce026e-4bf1-4f67-b72f-b2799612b36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:57.213531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b16696-e5b1-4c04-a9e3-2ea137bc728d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.214310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b16696-e5b1-4c04-a9e3-2ea137bc728d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.214587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b16696-e5b1-4c04-a9e3-2ea137bc728d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.215249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce026e-4bf1-4f67-b72f-b2799612b36d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.215741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce026e-4bf1-4f67-b72f-b2799612b36d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.216499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce026e-4bf1-4f67-b72f-b2799612b36d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.217556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce026e-4bf1-4f67-b72f-b2799612b36d None admin] RBAC: Adding query filter params (group.id=4d5b6821203d4f5689ab71ea92833abb, role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=811b356a854c422f85cddd17b5c478aa) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:57.218076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce026e-4bf1-4f67-b72f-b2799612b36d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:57.219714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce026e-4bf1-4f67-b72f-b2799612b36d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:57.220137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 183/358] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:06:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:06:57.220942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b16696-e5b1-4c04-a9e3-2ea137bc728d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:57.232172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 176/359] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:06:57 2022] GET /identity/v3/role_assignments?group.id=4d5b6821203d4f5689ab71ea92833abb&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:57.233403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce026e-4bf1-4f67-b72f-b2799612b36d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:57.353126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce026e-4bf1-4f67-b72f-b2799612b36d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:57.359290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b16696-e5b1-4c04-a9e3-2ea137bc728d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:57.380363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:57.403307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40474d69-e82c-4792-881f-9103771a64d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:57.404027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40474d69-e82c-4792-881f-9103771a64d2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.404309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40474d69-e82c-4792-881f-9103771a64d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.404583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40474d69-e82c-4792-881f-9103771a64d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.413589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c564a0-c426-4131-82d3-8d2578ac1f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:57.414638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c564a0-c426-4131-82d3-8d2578ac1f16 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.414923 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c564a0-c426-4131-82d3-8d2578ac1f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.415202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c564a0-c426-4131-82d3-8d2578ac1f16 None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.417842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c564a0-c426-4131-82d3-8d2578ac1f16 None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:57.419327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c564a0-c426-4131-82d3-8d2578ac1f16 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:57.421118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 184/360] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:06:57 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:57.421814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c564a0-c426-4131-82d3-8d2578ac1f16 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:57.467798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:57.493341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40474d69-e82c-4792-881f-9103771a64d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:57.494822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40474d69-e82c-4792-881f-9103771a64d2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:57.552647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c564a0-c426-4131-82d3-8d2578ac1f16 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:57.596772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87390b63-4a70-4f7f-b310-af2ec8b091e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:57.597479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87390b63-4a70-4f7f-b310-af2ec8b091e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.597765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87390b63-4a70-4f7f-b310-af2ec8b091e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.598040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87390b63-4a70-4f7f-b310-af2ec8b091e2 None admin] PATH_INFO: `/v3/users/b262eee2b1d946d388511b5d27a9f5fc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.599705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87390b63-4a70-4f7f-b310-af2ec8b091e2 None admin] RBAC: Authorizing `identity:get_user(user_id=b262eee2b1d946d388511b5d27a9f5fc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:57.602053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87390b63-4a70-4f7f-b310-af2ec8b091e2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:57.605241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 185/361] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:06:57 2022] GET /identity/v3/users/b262eee2b1d946d388511b5d27a9f5fc => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:57.605773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aac833-dd09-436c-8fe8-ae4d60cce3c1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.606057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aac833-dd09-436c-8fe8-ae4d60cce3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.606438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aac833-dd09-436c-8fe8-ae4d60cce3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.627999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-42aac833-dd09-436c-8fe8-ae4d60cce3c1 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:06:57.630807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-42aac833-dd09-436c-8fe8-ae4d60cce3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:06:57.640971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 177/362] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:57.642029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40474d69-e82c-4792-881f-9103771a64d2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:57.668819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:57.721192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 186/363] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:06:57.722158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aac833-dd09-436c-8fe8-ae4d60cce3c1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:57.762994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:57.793221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40474d69-e82c-4792-881f-9103771a64d2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:57.851158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10ee26-36ea-454e-a3f8-5d5227e57e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:57.851830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ee26-36ea-454e-a3f8-5d5227e57e94 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.852113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ee26-36ea-454e-a3f8-5d5227e57e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.852442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10ee26-36ea-454e-a3f8-5d5227e57e94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.854525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aac833-dd09-436c-8fe8-ae4d60cce3c1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:57.864376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10ee26-36ea-454e-a3f8-5d5227e57e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3e10ee26-36ea-454e-a3f8-5d5227e57e94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:57.870363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:57.871237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 178/364] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:06:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:57.876187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10ee26-36ea-454e-a3f8-5d5227e57e94 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:57.904014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1427cb0-3d3f-4163-9d8e-50cd4d198769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:57.904882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1427cb0-3d3f-4163-9d8e-50cd4d198769 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:57.908689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1427cb0-3d3f-4163-9d8e-50cd4d198769 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:57.909182 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1427cb0-3d3f-4163-9d8e-50cd4d198769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:57.910135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1427cb0-3d3f-4163-9d8e-50cd4d198769 None admin] RBAC: Adding query filter params (group.id=4d5b6821203d4f5689ab71ea92833abb, role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=811b356a854c422f85cddd17b5c478aa) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:57.910629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1427cb0-3d3f-4163-9d8e-50cd4d198769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:57.912373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1427cb0-3d3f-4163-9d8e-50cd4d198769 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:57.918709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 187/365] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:06:57 2022] GET /identity/v3/role_assignments?group.id=4d5b6821203d4f5689ab71ea92833abb&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:57.919926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1427cb0-3d3f-4163-9d8e-50cd4d198769 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:57.990636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:58.040364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10ee26-36ea-454e-a3f8-5d5227e57e94 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:58.072059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:58.083854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1427cb0-3d3f-4163-9d8e-50cd4d198769 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:58.100614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052f784-1f54-4078-ade4-1184f4eecf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:58.101126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f784-1f54-4078-ade4-1184f4eecf8b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:58.101408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f784-1f54-4078-ade4-1184f4eecf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:58.101676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052f784-1f54-4078-ade4-1184f4eecf8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:58.102405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052f784-1f54-4078-ade4-1184f4eecf8b None admin] RBAC: Adding query filter params (role.id=9dbcf5e8a10a4728888b9f4e923f5814, user.id=b262eee2b1d946d388511b5d27a9f5fc, scope.system=all) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:58.102707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052f784-1f54-4078-ade4-1184f4eecf8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:58.104267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052f784-1f54-4078-ade4-1184f4eecf8b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:58.125179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 179/366] 10.222.0.71 () {56 vars in 1308 bytes} [Wed Sep 21 12:06:57 2022] GET /identity/v3/role_assignments?user.id=b262eee2b1d946d388511b5d27a9f5fc&scope.system=all&role.id=9dbcf5e8a10a4728888b9f4e923f5814 => generated 510 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:58.125179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052f784-1f54-4078-ade4-1184f4eecf8b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:58.139304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775a386-02e8-40cc-a07c-da06114be8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:58.140011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775a386-02e8-40cc-a07c-da06114be8ce None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:58.140287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775a386-02e8-40cc-a07c-da06114be8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:58.140554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775a386-02e8-40cc-a07c-da06114be8ce None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:58.142759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775a386-02e8-40cc-a07c-da06114be8ce None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:58.144357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775a386-02e8-40cc-a07c-da06114be8ce None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:58.146205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 188/367] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:57 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:58.151798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775a386-02e8-40cc-a07c-da06114be8ce None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:58.283295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775a386-02e8-40cc-a07c-da06114be8ce None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:58.290717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052f784-1f54-4078-ade4-1184f4eecf8b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:58.334892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25541f-e55d-4677-a51c-c57cda40ecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:58.335532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25541f-e55d-4677-a51c-c57cda40ecfb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:58.335848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25541f-e55d-4677-a51c-c57cda40ecfb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:58.336125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25541f-e55d-4677-a51c-c57cda40ecfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:58.337787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151804c9-7367-4360-96e8-cf23a5523a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:58.340101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151804c9-7367-4360-96e8-cf23a5523a70 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:58.340379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151804c9-7367-4360-96e8-cf23a5523a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:58.340646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151804c9-7367-4360-96e8-cf23a5523a70 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:58.341584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151804c9-7367-4360-96e8-cf23a5523a70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:58.341879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151804c9-7367-4360-96e8-cf23a5523a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:58.343281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151804c9-7367-4360-96e8-cf23a5523a70 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:58.353119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:58.356240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 180/368] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:06:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:58.357665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151804c9-7367-4360-96e8-cf23a5523a70 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:58.364420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25541f-e55d-4677-a51c-c57cda40ecfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ac25541f-e55d-4677-a51c-c57cda40ecfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:06:58.370506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:06:58.371417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:06:58.372230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:06:58.376483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 189/369] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:06:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:06:58.377505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25541f-e55d-4677-a51c-c57cda40ecfb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:58.416827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:58.502637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151804c9-7367-4360-96e8-cf23a5523a70 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:58.525651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25541f-e55d-4677-a51c-c57cda40ecfb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:58.543681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbe1b7-3412-41f8-88f3-6f108bf0c97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:58.544296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbe1b7-3412-41f8-88f3-6f108bf0c97d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:58.544533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbe1b7-3412-41f8-88f3-6f108bf0c97d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:58.544801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbe1b7-3412-41f8-88f3-6f108bf0c97d None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:58.545914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbe1b7-3412-41f8-88f3-6f108bf0c97d None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:58.547523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbe1b7-3412-41f8-88f3-6f108bf0c97d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:58.552500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 181/370] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:06:58 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:58.552500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbe1b7-3412-41f8-88f3-6f108bf0c97d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:58.580598 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c262ff9-63bb-4316-887a-e187d9e349d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:58.582466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c262ff9-63bb-4316-887a-e187d9e349d7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:58.584055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c262ff9-63bb-4316-887a-e187d9e349d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:58.584916 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c262ff9-63bb-4316-887a-e187d9e349d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:58.646550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:58.678794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbe1b7-3412-41f8-88f3-6f108bf0c97d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:58.720701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e10e8-d67b-4695-a1f2-1684f8646c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:58.721440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e10e8-d67b-4695-a1f2-1684f8646c9e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:58.721780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e10e8-d67b-4695-a1f2-1684f8646c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:58.722113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e10e8-d67b-4695-a1f2-1684f8646c9e None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:58.722847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e10e8-d67b-4695-a1f2-1684f8646c9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:58.723254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e10e8-d67b-4695-a1f2-1684f8646c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:58.725080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e10e8-d67b-4695-a1f2-1684f8646c9e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:58.725392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c262ff9-63bb-4316-887a-e187d9e349d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:58.728569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c262ff9-63bb-4316-887a-e187d9e349d7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:58.737213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 182/371] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:06:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:58.738362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e10e8-d67b-4695-a1f2-1684f8646c9e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:58.784234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:58.855653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e10e8-d67b-4695-a1f2-1684f8646c9e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:58.889408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:58.902660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fe97c-e58d-457c-8e81-025098a99f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:58.903513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fe97c-e58d-457c-8e81-025098a99f12 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:58.903926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fe97c-e58d-457c-8e81-025098a99f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:58.904292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fe97c-e58d-457c-8e81-025098a99f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:58.905419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fe97c-e58d-457c-8e81-025098a99f12 None admin] RBAC: Adding query filter params (group.id=4d5b6821203d4f5689ab71ea92833abb, role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=a06126b4287647ca9294ce703dd39691) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:58.905824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fe97c-e58d-457c-8e81-025098a99f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:58.907366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fe97c-e58d-457c-8e81-025098a99f12 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:58.914508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 183/372] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:06:58 2022] GET /identity/v3/role_assignments?group.id=4d5b6821203d4f5689ab71ea92833abb&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:58.916402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fe97c-e58d-457c-8e81-025098a99f12 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:58.941758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 190/373] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:58.948092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c262ff9-63bb-4316-887a-e187d9e349d7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:59.056115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fe97c-e58d-457c-8e81-025098a99f12 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:59.075915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c262ff9-63bb-4316-887a-e187d9e349d7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:59.103083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c4a5-f3f1-4501-8ac2-e2c7cb032cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:59.104728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c4a5-f3f1-4501-8ac2-e2c7cb032cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:59.105224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c4a5-f3f1-4501-8ac2-e2c7cb032cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:59.105752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4c4a5-f3f1-4501-8ac2-e2c7cb032cd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:59.107522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:59.136311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d893a-90be-4566-9f92-c854f14197ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:59.136964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d893a-90be-4566-9f92-c854f14197ef None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:59.137239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d893a-90be-4566-9f92-c854f14197ef None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:59.137504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d893a-90be-4566-9f92-c854f14197ef None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:59.140284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d893a-90be-4566-9f92-c854f14197ef None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:59.142074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d893a-90be-4566-9f92-c854f14197ef None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:59.145022 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 191/374] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:58 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:59.150498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d893a-90be-4566-9f92-c854f14197ef None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:59.224354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:59.234994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4c4a5-f3f1-4501-8ac2-e2c7cb032cd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:59.237873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4c4a5-f3f1-4501-8ac2-e2c7cb032cd2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:59.308644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d893a-90be-4566-9f92-c854f14197ef None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:59.380383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e24f2-6b84-4116-85c9-b72122165549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:59.382049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e24f2-6b84-4116-85c9-b72122165549 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:59.382663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e24f2-6b84-4116-85c9-b72122165549 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:59.383732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e24f2-6b84-4116-85c9-b72122165549 None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:59.386777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e24f2-6b84-4116-85c9-b72122165549 None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:59.388472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e24f2-6b84-4116-85c9-b72122165549 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:59.391422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 192/375] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:06:59 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:59.395078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 184/376] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:06:59.399017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e24f2-6b84-4116-85c9-b72122165549 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:59.400952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c4a5-f3f1-4501-8ac2-e2c7cb032cd2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:59.508421 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:59.587909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:59.592903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e24f2-6b84-4116-85c9-b72122165549 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:59.607413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4c4a5-f3f1-4501-8ac2-e2c7cb032cd2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:59.663595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1332af-1ae1-4968-a5fc-06d3d3c91fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:59.664145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1332af-1ae1-4968-a5fc-06d3d3c91fca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:59.664425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1332af-1ae1-4968-a5fc-06d3d3c91fca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:59.664696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1332af-1ae1-4968-a5fc-06d3d3c91fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:59.665544 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1332af-1ae1-4968-a5fc-06d3d3c91fca None admin] RBAC: Adding query filter params (group.id=4d5b6821203d4f5689ab71ea92833abb, role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=a06126b4287647ca9294ce703dd39691) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:06:59.665856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1332af-1ae1-4968-a5fc-06d3d3c91fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:59.666217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fd0c3-5fca-4e60-b107-7fed90918a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:59.667629 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fd0c3-5fca-4e60-b107-7fed90918a3d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:59.667867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1332af-1ae1-4968-a5fc-06d3d3c91fca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:59.669291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fd0c3-5fca-4e60-b107-7fed90918a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:59.669820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fd0c3-5fca-4e60-b107-7fed90918a3d None admin] PATH_INFO: `/v3/projects/5d4deb4c3b0e444f92c029c040b5603c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:59.676625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fd0c3-5fca-4e60-b107-7fed90918a3d None admin] RBAC: Authorizing `identity:get_project(project_id=5d4deb4c3b0e444f92c029c040b5603c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:59.678352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fd0c3-5fca-4e60-b107-7fed90918a3d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:59.680285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 185/377] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:06:59 2022] GET /identity/v3/projects/5d4deb4c3b0e444f92c029c040b5603c => generated 344 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:59.680676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 193/378] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:06:59 2022] GET /identity/v3/role_assignments?group.id=4d5b6821203d4f5689ab71ea92833abb&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 260 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:06:59.687327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1332af-1ae1-4968-a5fc-06d3d3c91fca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:06:59.851943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1332af-1ae1-4968-a5fc-06d3d3c91fca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:06:59.882871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:06:59.902608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6923d-3c07-4f9d-a2c1-47b2c8656e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:06:59.903300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6923d-3c07-4f9d-a2c1-47b2c8656e74 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:06:59.903578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6923d-3c07-4f9d-a2c1-47b2c8656e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:06:59.903885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6923d-3c07-4f9d-a2c1-47b2c8656e74 None admin] PATH_INFO: `/v3/groups/9fabaa52b48e4c61868df76149a0a2b4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:06:59.909366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6923d-3c07-4f9d-a2c1-47b2c8656e74 None admin] RBAC: Authorizing `identity:get_group(group_id=9fabaa52b48e4c61868df76149a0a2b4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:06:59.910850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6923d-3c07-4f9d-a2c1-47b2c8656e74 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:06:59.912882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 194/379] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:06:59 2022] GET /identity/v3/groups/9fabaa52b48e4c61868df76149a0a2b4 => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:06:59.918626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fd0c3-5fca-4e60-b107-7fed90918a3d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:00.059131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fd0c3-5fca-4e60-b107-7fed90918a3d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:00.101721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e586c4-5b90-4692-9fc9-f9a74da4f51e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:00.102035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e586c4-5b90-4692-9fc9-f9a74da4f51e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:00.102302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e586c4-5b90-4692-9fc9-f9a74da4f51e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:00.103919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 195/380] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:00.110588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadb56d-c2dc-4931-9f81-08c8c2173abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:00.110775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29c699-6ebc-451e-807b-d1270e54affa None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:00.110979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29c699-6ebc-451e-807b-d1270e54affa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:00.111203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadb56d-c2dc-4931-9f81-08c8c2173abe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:00.111307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29c699-6ebc-451e-807b-d1270e54affa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:00.111477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadb56d-c2dc-4931-9f81-08c8c2173abe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:00.111741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadb56d-c2dc-4931-9f81-08c8c2173abe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:00.112486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadb56d-c2dc-4931-9f81-08c8c2173abe None admin] RBAC: Adding query filter params (group.id=9fabaa52b48e4c61868df76149a0a2b4, role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=5d4deb4c3b0e444f92c029c040b5603c) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:00.112788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadb56d-c2dc-4931-9f81-08c8c2173abe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:00.114275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadb56d-c2dc-4931-9f81-08c8c2173abe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:00.120547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 186/381] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:06:59 2022] GET /identity/v3/role_assignments?group.id=9fabaa52b48e4c61868df76149a0a2b4&scope.project.id=5d4deb4c3b0e444f92c029c040b5603c&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:00.145049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1e29c699-6ebc-451e-807b-d1270e54affa None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:00.146290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1e29c699-6ebc-451e-807b-d1270e54affa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:00.269832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 196/382] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:00.695917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadb56d-c2dc-4931-9f81-08c8c2173abe None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:00.737355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:00.832436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadb56d-c2dc-4931-9f81-08c8c2173abe None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:00.870773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d040a-52ad-4224-83c0-6af80ffc09af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:00.871415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d040a-52ad-4224-83c0-6af80ffc09af None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:00.871692 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d040a-52ad-4224-83c0-6af80ffc09af None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:00.872002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d040a-52ad-4224-83c0-6af80ffc09af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:00.882472 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d040a-52ad-4224-83c0-6af80ffc09af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-050d040a-52ad-4224-83c0-6af80ffc09af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:00.888642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:00.889744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:00.890707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 187/383] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:00.897407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29c699-6ebc-451e-807b-d1270e54affa None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:00.993429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a9c0d-8846-4ac1-b4e2-c119ae7fbafe None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:00.993670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a9c0d-8846-4ac1-b4e2-c119ae7fbafe None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:00.993893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a9c0d-8846-4ac1-b4e2-c119ae7fbafe None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:00.995388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 188/384] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:01.000390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c728eb2-221c-423a-9097-dea6b67330df None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:01.000679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c728eb2-221c-423a-9097-dea6b67330df None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:01.000955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c728eb2-221c-423a-9097-dea6b67330df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:01.007745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29c699-6ebc-451e-807b-d1270e54affa None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:01.026639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9c728eb2-221c-423a-9097-dea6b67330df None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:01.027957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9c728eb2-221c-423a-9097-dea6b67330df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:01.038558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:01.075431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7679b-9de4-40a4-bd8b-ef54f06de2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:01.076079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7679b-9de4-40a4-bd8b-ef54f06de2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:01.076357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7679b-9de4-40a4-bd8b-ef54f06de2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:01.076626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7679b-9de4-40a4-bd8b-ef54f06de2f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:01.077295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7679b-9de4-40a4-bd8b-ef54f06de2f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:01.077598 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7679b-9de4-40a4-bd8b-ef54f06de2f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:01.078977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7679b-9de4-40a4-bd8b-ef54f06de2f3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:01.096383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 197/385] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:07:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:01.102351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7679b-9de4-40a4-bd8b-ef54f06de2f3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:01.157153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 189/386] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:07:01.238158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7679b-9de4-40a4-bd8b-ef54f06de2f3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:01.273863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644ba0e-920e-49c5-9d9f-6c8e8eebfd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:01.274413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644ba0e-920e-49c5-9d9f-6c8e8eebfd5c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:01.274694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644ba0e-920e-49c5-9d9f-6c8e8eebfd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:01.274967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644ba0e-920e-49c5-9d9f-6c8e8eebfd5c None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:01.276726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644ba0e-920e-49c5-9d9f-6c8e8eebfd5c None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:01.278227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644ba0e-920e-49c5-9d9f-6c8e8eebfd5c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:01.280129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 198/387] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:01 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:01.289609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c728eb2-221c-423a-9097-dea6b67330df None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:01.328353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:01.449259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c728eb2-221c-423a-9097-dea6b67330df None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:01.495371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4819e-bdde-48e0-a080-dc1cfa6d95a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:01.496034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4819e-bdde-48e0-a080-dc1cfa6d95a8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:01.496306 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4819e-bdde-48e0-a080-dc1cfa6d95a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:01.496568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4819e-bdde-48e0-a080-dc1cfa6d95a8 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:01.500410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4819e-bdde-48e0-a080-dc1cfa6d95a8 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:01.501849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4819e-bdde-48e0-a080-dc1cfa6d95a8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:01.503837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 190/388] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:01 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:01.509642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644ba0e-920e-49c5-9d9f-6c8e8eebfd5c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:01.518054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4819e-bdde-48e0-a080-dc1cfa6d95a8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:01.598738 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:01.621044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:01.639914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4819e-bdde-48e0-a080-dc1cfa6d95a8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:01.677396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ed7c6-6c24-4bb6-8399-9e11b5c5b4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:01.678126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ed7c6-6c24-4bb6-8399-9e11b5c5b4a1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:01.678522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ed7c6-6c24-4bb6-8399-9e11b5c5b4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:01.678869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ed7c6-6c24-4bb6-8399-9e11b5c5b4a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:01.689970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ed7c6-6c24-4bb6-8399-9e11b5c5b4a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-aa3ed7c6-6c24-4bb6-8399-9e11b5c5b4a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:01.695328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:01.696687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:01.697529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 191/389] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:07:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:01.699174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a2a26-d0b0-4925-9973-466017d09073 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:01.699527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a2a26-d0b0-4925-9973-466017d09073 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:01.699902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a2a26-d0b0-4925-9973-466017d09073 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:01.701718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 192/390] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:01.704768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a2a26-d0b0-4925-9973-466017d09073 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:01.714280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644ba0e-920e-49c5-9d9f-6c8e8eebfd5c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:01.752018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711117a8-7bcd-41c7-a927-0eb8d7e9dfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:01.752811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711117a8-7bcd-41c7-a927-0eb8d7e9dfe0 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:01.753179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711117a8-7bcd-41c7-a927-0eb8d7e9dfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:01.753521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711117a8-7bcd-41c7-a927-0eb8d7e9dfe0 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa/groups/4d5b6821203d4f5689ab71ea92833abb/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:01.755586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711117a8-7bcd-41c7-a927-0eb8d7e9dfe0 None admin] RBAC: Authorizing `identity:create_grant(project_id=811b356a854c422f85cddd17b5c478aa, group_id=4d5b6821203d4f5689ab71ea92833abb, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:01.758155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711117a8-7bcd-41c7-a927-0eb8d7e9dfe0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:01.777777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 199/391] 10.222.0.71 () {58 vars in 1442 bytes} [Wed Sep 21 12:07:01 2022] PUT /identity/v3/projects/811b356a854c422f85cddd17b5c478aa/groups/4d5b6821203d4f5689ab71ea92833abb/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:01.780218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab69809-0c8f-4496-891d-fbdbf68bcc05 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:01.781063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab69809-0c8f-4496-891d-fbdbf68bcc05 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:01.781605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab69809-0c8f-4496-891d-fbdbf68bcc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:01.809283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ab69809-0c8f-4496-891d-fbdbf68bcc05 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:01.811788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ab69809-0c8f-4496-891d-fbdbf68bcc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:01.839795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a2a26-d0b0-4925-9973-466017d09073 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:01.921113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa92a1a-77d9-4a0b-a287-a2ff58d8f56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:01.921796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa92a1a-77d9-4a0b-a287-a2ff58d8f56e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:01.922078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa92a1a-77d9-4a0b-a287-a2ff58d8f56e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:01.922348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa92a1a-77d9-4a0b-a287-a2ff58d8f56e None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:01.924136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa92a1a-77d9-4a0b-a287-a2ff58d8f56e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:01.924445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa92a1a-77d9-4a0b-a287-a2ff58d8f56e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:01.925968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa92a1a-77d9-4a0b-a287-a2ff58d8f56e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:01.949944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 200/392] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:01.950517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 193/393] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:01.957036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab69809-0c8f-4496-891d-fbdbf68bcc05 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:01.997132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:02.080881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab69809-0c8f-4496-891d-fbdbf68bcc05 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:02.129808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bd0ff-315f-475c-8c12-ac8032e4844d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:02.130449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bd0ff-315f-475c-8c12-ac8032e4844d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:02.130729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bd0ff-315f-475c-8c12-ac8032e4844d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:02.131005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8bd0ff-315f-475c-8c12-ac8032e4844d None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:02.132240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8bd0ff-315f-475c-8c12-ac8032e4844d None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:02.133679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8bd0ff-315f-475c-8c12-ac8032e4844d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:02.135628 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 201/394] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:01 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:02.141292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa92a1a-77d9-4a0b-a287-a2ff58d8f56e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:02.156884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:02.291565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bd0ff-315f-475c-8c12-ac8032e4844d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:02.301674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa92a1a-77d9-4a0b-a287-a2ff58d8f56e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:02.348531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868d74d-a421-4693-9302-665b7b3ba6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:02.349172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868d74d-a421-4693-9302-665b7b3ba6fc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:02.349449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868d74d-a421-4693-9302-665b7b3ba6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:02.349717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868d74d-a421-4693-9302-665b7b3ba6fc None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:02.351005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868d74d-a421-4693-9302-665b7b3ba6fc None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:02.353324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868d74d-a421-4693-9302-665b7b3ba6fc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:02.361423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 194/395] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:02 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:02.370681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868d74d-a421-4693-9302-665b7b3ba6fc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:02.508872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8bd0ff-315f-475c-8c12-ac8032e4844d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:02.510880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:02.517643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:02.547544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868d74d-a421-4693-9302-665b7b3ba6fc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:02.556811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d5492-e7c3-4c3b-93cc-7f29efa54bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:02.557501 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d5492-e7c3-4c3b-93cc-7f29efa54bbf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:02.557780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d5492-e7c3-4c3b-93cc-7f29efa54bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:02.558051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d5492-e7c3-4c3b-93cc-7f29efa54bbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:02.569003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d5492-e7c3-4c3b-93cc-7f29efa54bbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-409d5492-e7c3-4c3b-93cc-7f29efa54bbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:02.574063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:02.574946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:02.576245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 202/396] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:02.576890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeaecf-5dd0-4442-a843-7fb3f2c9560e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:02.577189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeaecf-5dd0-4442-a843-7fb3f2c9560e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:02.577278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeaecf-5dd0-4442-a843-7fb3f2c9560e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:02.579795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 203/397] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:02.583232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baeaecf-5dd0-4442-a843-7fb3f2c9560e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:02.591385 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ea0e8-8b54-4d60-b384-c3cfe13a03f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:02.592147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ea0e8-8b54-4d60-b384-c3cfe13a03f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:02.592431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ea0e8-8b54-4d60-b384-c3cfe13a03f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:02.592709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ea0e8-8b54-4d60-b384-c3cfe13a03f4 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa/groups/4d5b6821203d4f5689ab71ea92833abb/roles/23d9f4769fd6424bae83090a691c7f94` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:02.595221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ea0e8-8b54-4d60-b384-c3cfe13a03f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=811b356a854c422f85cddd17b5c478aa, group_id=4d5b6821203d4f5689ab71ea92833abb, role_id=23d9f4769fd6424bae83090a691c7f94)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:02.597257 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ea0e8-8b54-4d60-b384-c3cfe13a03f4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:02.623122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 195/398] 10.222.0.71 () {58 vars in 1442 bytes} [Wed Sep 21 12:07:02 2022] PUT /identity/v3/projects/811b356a854c422f85cddd17b5c478aa/groups/4d5b6821203d4f5689ab71ea92833abb/roles/23d9f4769fd6424bae83090a691c7f94 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:07:02.630315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f564aaf-fd67-4a7b-8d2c-6c9197f94af1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:02.630315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f564aaf-fd67-4a7b-8d2c-6c9197f94af1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:02.630315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f564aaf-fd67-4a7b-8d2c-6c9197f94af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:02.654905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0f564aaf-fd67-4a7b-8d2c-6c9197f94af1 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:02.656267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0f564aaf-fd67-4a7b-8d2c-6c9197f94af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:02.741087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 196/399] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:02.796106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baeaecf-5dd0-4442-a843-7fb3f2c9560e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:02.834109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aa90d-9369-4151-b978-03fa42507f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:02.834790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aa90d-9369-4151-b978-03fa42507f24 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:02.835076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aa90d-9369-4151-b978-03fa42507f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:02.835358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aa90d-9369-4151-b978-03fa42507f24 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:02.836067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99aa90d-9369-4151-b978-03fa42507f24 None admin] RBAC: Adding query filter params (name=member) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:02.836614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99aa90d-9369-4151-b978-03fa42507f24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:02.838257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99aa90d-9369-4151-b978-03fa42507f24 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:02.849622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 204/400] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:07:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:02.855592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f564aaf-fd67-4a7b-8d2c-6c9197f94af1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:02.896616 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293a720-a2e8-4ebe-9834-ec0ac889ae41 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:02.897096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293a720-a2e8-4ebe-9834-ec0ac889ae41 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:02.897379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293a720-a2e8-4ebe-9834-ec0ac889ae41 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:02.898925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 205/401] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:02.915337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4064d3-abe8-4b9c-afd9-c46f12a4db5a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:02.915653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4064d3-abe8-4b9c-afd9-c46f12a4db5a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:02.916021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4064d3-abe8-4b9c-afd9-c46f12a4db5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:02.944012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:02.947648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1a4064d3-abe8-4b9c-afd9-c46f12a4db5a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:02.951572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1a4064d3-abe8-4b9c-afd9-c46f12a4db5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:02.984137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:02.995428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f564aaf-fd67-4a7b-8d2c-6c9197f94af1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:03.038859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c565ab-c9ef-4399-9ff3-bac1a4d2c83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:03.040113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c565ab-c9ef-4399-9ff3-bac1a4d2c83c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:03.041078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c565ab-c9ef-4399-9ff3-bac1a4d2c83c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:03.041583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c565ab-c9ef-4399-9ff3-bac1a4d2c83c None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:03.044806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c565ab-c9ef-4399-9ff3-bac1a4d2c83c None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:03.046704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c565ab-c9ef-4399-9ff3-bac1a4d2c83c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:03.048893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 197/402] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:02 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:03.054413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c565ab-c9ef-4399-9ff3-bac1a4d2c83c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:03.059582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 206/403] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 21 12:07:03.090794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4064d3-abe8-4b9c-afd9-c46f12a4db5a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:03.219371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c565ab-c9ef-4399-9ff3-bac1a4d2c83c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:03.275365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4064d3-abe8-4b9c-afd9-c46f12a4db5a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:03.283054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:03.296458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e82f3-42cd-48d4-8c42-421a1b20f0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:03.297170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e82f3-42cd-48d4-8c42-421a1b20f0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:03.297446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e82f3-42cd-48d4-8c42-421a1b20f0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:03.297717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e82f3-42cd-48d4-8c42-421a1b20f0f0 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:03.299254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e82f3-42cd-48d4-8c42-421a1b20f0f0 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:03.304253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e82f3-42cd-48d4-8c42-421a1b20f0f0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:03.305908 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 198/404] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:03 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:03.313645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e82f3-42cd-48d4-8c42-421a1b20f0f0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:03.356765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08642c-969d-4f3c-a499-46111d73d458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:03.357357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08642c-969d-4f3c-a499-46111d73d458 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:03.357681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08642c-969d-4f3c-a499-46111d73d458 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:03.357966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08642c-969d-4f3c-a499-46111d73d458 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:03.383430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08642c-969d-4f3c-a499-46111d73d458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0c08642c-969d-4f3c-a499-46111d73d458 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:03.389114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:03.389995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:03.390772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 207/405] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:07:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:03.397842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08642c-969d-4f3c-a499-46111d73d458 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:03.422524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:03.468419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e82f3-42cd-48d4-8c42-421a1b20f0f0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:03.512727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e95c0-9baa-4f22-8aa7-b9444c4a05d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:03.513859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e95c0-9baa-4f22-8aa7-b9444c4a05d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:03.514142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e95c0-9baa-4f22-8aa7-b9444c4a05d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:03.514413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e95c0-9baa-4f22-8aa7-b9444c4a05d3 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691/groups/4d5b6821203d4f5689ab71ea92833abb/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:03.516861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e95c0-9baa-4f22-8aa7-b9444c4a05d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a06126b4287647ca9294ce703dd39691, group_id=4d5b6821203d4f5689ab71ea92833abb, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:03.518685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e95c0-9baa-4f22-8aa7-b9444c4a05d3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:03.537693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08642c-969d-4f3c-a499-46111d73d458 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:03.539830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 199/406] 10.222.0.71 () {58 vars in 1442 bytes} [Wed Sep 21 12:07:03 2022] PUT /identity/v3/projects/a06126b4287647ca9294ce703dd39691/groups/4d5b6821203d4f5689ab71ea92833abb/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:03.541093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e95c0-9baa-4f22-8aa7-b9444c4a05d3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:03.611615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97426fe-5614-48d9-9437-87f5c9e44ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:03.613887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97426fe-5614-48d9-9437-87f5c9e44ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:03.615207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97426fe-5614-48d9-9437-87f5c9e44ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:03.615894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97426fe-5614-48d9-9437-87f5c9e44ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:03.617790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97426fe-5614-48d9-9437-87f5c9e44ec2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:03.619292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97426fe-5614-48d9-9437-87f5c9e44ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:03.621747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97426fe-5614-48d9-9437-87f5c9e44ec2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:03.637587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 208/407] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:03.649073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97426fe-5614-48d9-9437-87f5c9e44ec2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:03.668154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:03.708135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e95c0-9baa-4f22-8aa7-b9444c4a05d3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:03.746899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c0852-a840-47ec-b39e-17c5b7d3f5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:03.748449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c0852-a840-47ec-b39e-17c5b7d3f5dd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:03.749001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c0852-a840-47ec-b39e-17c5b7d3f5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:03.749516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c0852-a840-47ec-b39e-17c5b7d3f5dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:03.770081 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c0852-a840-47ec-b39e-17c5b7d3f5dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a99c0852-a840-47ec-b39e-17c5b7d3f5dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:03.774990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:03.776562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:03.777369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 200/408] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:03.786486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c0852-a840-47ec-b39e-17c5b7d3f5dd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:03.802311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:03.852532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97426fe-5614-48d9-9437-87f5c9e44ec2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:03.900645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef723ea-18d2-41e3-90c3-45a65e549506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:03.901202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef723ea-18d2-41e3-90c3-45a65e549506 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:03.901482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef723ea-18d2-41e3-90c3-45a65e549506 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:03.901757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef723ea-18d2-41e3-90c3-45a65e549506 None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:03.903452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef723ea-18d2-41e3-90c3-45a65e549506 None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:03.905265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef723ea-18d2-41e3-90c3-45a65e549506 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:03.908028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 209/409] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:03 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:03.913241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef723ea-18d2-41e3-90c3-45a65e549506 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:03.922818 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c0852-a840-47ec-b39e-17c5b7d3f5dd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:03.959462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:03.989833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b88460-b12f-4298-bd5d-c41281ad31f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:03.990445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b88460-b12f-4298-bd5d-c41281ad31f2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:03.990723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b88460-b12f-4298-bd5d-c41281ad31f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:03.990997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b88460-b12f-4298-bd5d-c41281ad31f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:03.991663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b88460-b12f-4298-bd5d-c41281ad31f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:03.991987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b88460-b12f-4298-bd5d-c41281ad31f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:03.993380 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b88460-b12f-4298-bd5d-c41281ad31f2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:04.007518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 201/410] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:04.011363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b88460-b12f-4298-bd5d-c41281ad31f2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:04.094009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef723ea-18d2-41e3-90c3-45a65e549506 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:04.147782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77744de6-c7b4-4781-a6ac-bb8330a7238d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:04.147782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77744de6-c7b4-4781-a6ac-bb8330a7238d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:04.147782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77744de6-c7b4-4781-a6ac-bb8330a7238d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:04.147782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77744de6-c7b4-4781-a6ac-bb8330a7238d None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:04.148190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77744de6-c7b4-4781-a6ac-bb8330a7238d None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:04.148190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77744de6-c7b4-4781-a6ac-bb8330a7238d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:04.148318 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 210/411] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:03 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:04.154403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77744de6-c7b4-4781-a6ac-bb8330a7238d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:04.175917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:04.194720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b88460-b12f-4298-bd5d-c41281ad31f2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:04.260745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81badb7-40dd-467e-899a-dad87f6ea355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:04.261406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81badb7-40dd-467e-899a-dad87f6ea355 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:04.261681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81badb7-40dd-467e-899a-dad87f6ea355 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:04.261952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81badb7-40dd-467e-899a-dad87f6ea355 None admin] PATH_INFO: `/v3/groups/9fabaa52b48e4c61868df76149a0a2b4` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:04.263980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81badb7-40dd-467e-899a-dad87f6ea355 None admin] RBAC: Authorizing `identity:get_group(group_id=9fabaa52b48e4c61868df76149a0a2b4)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:04.265437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81badb7-40dd-467e-899a-dad87f6ea355 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:04.268609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 202/412] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:04 2022] GET /identity/v3/groups/9fabaa52b48e4c61868df76149a0a2b4 => generated 229 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:04.272766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81badb7-40dd-467e-899a-dad87f6ea355 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:04.327183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77744de6-c7b4-4781-a6ac-bb8330a7238d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:04.368976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:04.398818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e946b04-f773-4bb2-8253-41df75163998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:04.399489 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e946b04-f773-4bb2-8253-41df75163998 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:04.399783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e946b04-f773-4bb2-8253-41df75163998 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:04.400064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e946b04-f773-4bb2-8253-41df75163998 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691/groups/4d5b6821203d4f5689ab71ea92833abb/roles/23d9f4769fd6424bae83090a691c7f94` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:04.404983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e946b04-f773-4bb2-8253-41df75163998 None admin] RBAC: Authorizing `identity:create_grant(project_id=a06126b4287647ca9294ce703dd39691, group_id=4d5b6821203d4f5689ab71ea92833abb, role_id=23d9f4769fd6424bae83090a691c7f94)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:04.406445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e946b04-f773-4bb2-8253-41df75163998 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:04.422283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 211/413] 10.222.0.71 () {58 vars in 1442 bytes} [Wed Sep 21 12:07:04 2022] PUT /identity/v3/projects/a06126b4287647ca9294ce703dd39691/groups/4d5b6821203d4f5689ab71ea92833abb/roles/23d9f4769fd6424bae83090a691c7f94 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:07:04.441242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16688dc0-6903-4cb9-a6fb-3db398085d80 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:04.441546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16688dc0-6903-4cb9-a6fb-3db398085d80 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:04.441803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16688dc0-6903-4cb9-a6fb-3db398085d80 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:04.443354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 212/414] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:04.450028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82320a5-4a53-4abb-ad3f-e6b2d23c2c94 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:04.450643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82320a5-4a53-4abb-ad3f-e6b2d23c2c94 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:04.451137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82320a5-4a53-4abb-ad3f-e6b2d23c2c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:04.468689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81badb7-40dd-467e-899a-dad87f6ea355 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:04.473301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f82320a5-4a53-4abb-ad3f-e6b2d23c2c94 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:04.474856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f82320a5-4a53-4abb-ad3f-e6b2d23c2c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:04.513562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aea42-a5df-4fdf-9dff-c6aa09a54148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:04.513562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aea42-a5df-4fdf-9dff-c6aa09a54148 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:04.513832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aea42-a5df-4fdf-9dff-c6aa09a54148 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:04.514110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aea42-a5df-4fdf-9dff-c6aa09a54148 None admin] PATH_INFO: `/v3/projects/5d4deb4c3b0e444f92c029c040b5603c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:04.517643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8aea42-a5df-4fdf-9dff-c6aa09a54148 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4deb4c3b0e444f92c029c040b5603c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:04.520254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8aea42-a5df-4fdf-9dff-c6aa09a54148 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:04.524677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 203/415] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:04 2022] GET /identity/v3/projects/5d4deb4c3b0e444f92c029c040b5603c => generated 344 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:04.531614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aea42-a5df-4fdf-9dff-c6aa09a54148 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:04.617185 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 213/416] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:07:04.689422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aea42-a5df-4fdf-9dff-c6aa09a54148 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:04.707345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:04.729926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36130f08-f5d2-48bb-a785-75e056d1f408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:04.730603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36130f08-f5d2-48bb-a785-75e056d1f408 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:04.730885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36130f08-f5d2-48bb-a785-75e056d1f408 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:04.731158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36130f08-f5d2-48bb-a785-75e056d1f408 None admin] PATH_INFO: `/v3/projects/5d4deb4c3b0e444f92c029c040b5603c/groups/9fabaa52b48e4c61868df76149a0a2b4/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:04.733114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36130f08-f5d2-48bb-a785-75e056d1f408 None admin] RBAC: Authorizing `identity:create_grant(project_id=5d4deb4c3b0e444f92c029c040b5603c, group_id=9fabaa52b48e4c61868df76149a0a2b4, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:04.734558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36130f08-f5d2-48bb-a785-75e056d1f408 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:04.755815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 204/417] 10.222.0.71 () {58 vars in 1442 bytes} [Wed Sep 21 12:07:04 2022] PUT /identity/v3/projects/5d4deb4c3b0e444f92c029c040b5603c/groups/9fabaa52b48e4c61868df76149a0a2b4/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:05.047516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82320a5-4a53-4abb-ad3f-e6b2d23c2c94 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:05.049905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:05.180384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82320a5-4a53-4abb-ad3f-e6b2d23c2c94 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:05.217722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb34f2c-166e-4e14-897c-dc1d00fd4bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:05.218423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb34f2c-166e-4e14-897c-dc1d00fd4bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:05.218709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb34f2c-166e-4e14-897c-dc1d00fd4bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:05.218981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb34f2c-166e-4e14-897c-dc1d00fd4bd6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:05.230833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb34f2c-166e-4e14-897c-dc1d00fd4bd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0eb34f2c-166e-4e14-897c-dc1d00fd4bd6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:05.235786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:05.236670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:05.237591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 214/418] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:05.245382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36130f08-f5d2-48bb-a785-75e056d1f408 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:05.333340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ea804-a29b-4eb1-bdae-0ed3cf0b8179 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:05.333697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ea804-a29b-4eb1-bdae-0ed3cf0b8179 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:05.333904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ea804-a29b-4eb1-bdae-0ed3cf0b8179 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:05.336135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 215/419] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:05.350269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d9180-e56f-44ca-9c77-182cb64c77b8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:05.350573 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d9180-e56f-44ca-9c77-182cb64c77b8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:05.350827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d9180-e56f-44ca-9c77-182cb64c77b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:05.383351 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36130f08-f5d2-48bb-a785-75e056d1f408 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:05.384807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a35d9180-e56f-44ca-9c77-182cb64c77b8 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:05.386634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a35d9180-e56f-44ca-9c77-182cb64c77b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:05.419300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d50fcf-dcab-4083-9042-717688784774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:05.419950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d50fcf-dcab-4083-9042-717688784774 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:05.420232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d50fcf-dcab-4083-9042-717688784774 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:05.420506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d50fcf-dcab-4083-9042-717688784774 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:05.421166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d50fcf-dcab-4083-9042-717688784774 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:05.421465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d50fcf-dcab-4083-9042-717688784774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:05.422848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d50fcf-dcab-4083-9042-717688784774 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:05.431664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 205/420] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:07:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:05.438333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d50fcf-dcab-4083-9042-717688784774 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:05.495778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:05.521131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:05.545412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 216/421] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:07:05.593343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d50fcf-dcab-4083-9042-717688784774 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:05.644510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1b828-90d4-4b4b-9733-da9c574fcdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:05.645135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1b828-90d4-4b4b-9733-da9c574fcdfe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:05.645474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1b828-90d4-4b4b-9733-da9c574fcdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:05.645744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1b828-90d4-4b4b-9733-da9c574fcdfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:05.722662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1b828-90d4-4b4b-9733-da9c574fcdfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:05.724138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1b828-90d4-4b4b-9733-da9c574fcdfe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:05.776670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:05.854824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 206/422] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 12:07:05.860706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d9180-e56f-44ca-9c77-182cb64c77b8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:05.933774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1b828-90d4-4b4b-9733-da9c574fcdfe None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:06.028071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d9180-e56f-44ca-9c77-182cb64c77b8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:06.094131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a22919-f16b-4a2d-92c8-bfd208ba51e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:06.096187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a22919-f16b-4a2d-92c8-bfd208ba51e3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:06.096971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a22919-f16b-4a2d-92c8-bfd208ba51e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:06.097430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a22919-f16b-4a2d-92c8-bfd208ba51e3 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:06.101048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a22919-f16b-4a2d-92c8-bfd208ba51e3 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:06.103799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a22919-f16b-4a2d-92c8-bfd208ba51e3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:06.110490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 217/423] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:05 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:06.113659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a22919-f16b-4a2d-92c8-bfd208ba51e3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:06.116761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1b828-90d4-4b4b-9733-da9c574fcdfe None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:06.189555 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841e860-dd50-466a-a467-7b32f4c66566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:06.190210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841e860-dd50-466a-a467-7b32f4c66566 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:06.190493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841e860-dd50-466a-a467-7b32f4c66566 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:06.190761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841e860-dd50-466a-a467-7b32f4c66566 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:06.205316 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:06.208375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5841e860-dd50-466a-a467-7b32f4c66566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5841e860-dd50-466a-a467-7b32f4c66566 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:06.215662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:06.216952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:06.217736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:06.220658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 207/424] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:07:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:06.224772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:06.227256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841e860-dd50-466a-a467-7b32f4c66566 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:06.254589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a22919-f16b-4a2d-92c8-bfd208ba51e3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:06.289859 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458c24a-f393-4f9a-bf38-f30b521abefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:06.290485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458c24a-f393-4f9a-bf38-f30b521abefe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:06.290763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458c24a-f393-4f9a-bf38-f30b521abefe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:06.291033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458c24a-f393-4f9a-bf38-f30b521abefe None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:06.292705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458c24a-f393-4f9a-bf38-f30b521abefe None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:06.294246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458c24a-f393-4f9a-bf38-f30b521abefe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:06.295998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 218/425] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:06 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:06.301668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458c24a-f393-4f9a-bf38-f30b521abefe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:06.399354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841e860-dd50-466a-a467-7b32f4c66566 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:06.421564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458c24a-f393-4f9a-bf38-f30b521abefe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:06.437193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a6d61-b6de-45ca-8c31-8bfab63b2e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:06.437797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6d61-b6de-45ca-8c31-8bfab63b2e11 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:06.438077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6d61-b6de-45ca-8c31-8bfab63b2e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:06.438454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6d61-b6de-45ca-8c31-8bfab63b2e11 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:06.439149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a6d61-b6de-45ca-8c31-8bfab63b2e11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:06.439522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a6d61-b6de-45ca-8c31-8bfab63b2e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:06.440955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a6d61-b6de-45ca-8c31-8bfab63b2e11 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:06.455413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:06.455790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 208/426] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:06.457828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c155d21-979e-4726-a85f-5d2f2b957ef9 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:06.458112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c155d21-979e-4726-a85f-5d2f2b957ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:06.458526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c155d21-979e-4726-a85f-5d2f2b957ef9 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:06.459947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 209/427] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:06.466200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c155d21-979e-4726-a85f-5d2f2b957ef9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:06.470392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a16021-ab80-48e7-846f-f52f23cc3ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:06.471235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a16021-ab80-48e7-846f-f52f23cc3ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:06.471586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a16021-ab80-48e7-846f-f52f23cc3ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:06.471965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a16021-ab80-48e7-846f-f52f23cc3ef5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:06.472770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a16021-ab80-48e7-846f-f52f23cc3ef5 None admin] RBAC: Adding query filter params (group.id=4d5b6821203d4f5689ab71ea92833abb, role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=811b356a854c422f85cddd17b5c478aa) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:06.473862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a16021-ab80-48e7-846f-f52f23cc3ef5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:06.474758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a16021-ab80-48e7-846f-f52f23cc3ef5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:06.481453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 219/428] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:07:06 2022] GET /identity/v3/role_assignments?group.id=4d5b6821203d4f5689ab71ea92833abb&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:06.482915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb5df-2d6c-4ad2-a392-d4fdaaafb3ff None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:06.483273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb5df-2d6c-4ad2-a392-d4fdaaafb3ff None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:06.483649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cb5df-2d6c-4ad2-a392-d4fdaaafb3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:06.505947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-814cb5df-2d6c-4ad2-a392-d4fdaaafb3ff None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:06.507429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-814cb5df-2d6c-4ad2-a392-d4fdaaafb3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:06.544479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:06.592524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c155d21-979e-4726-a85f-5d2f2b957ef9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:06.619201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 220/429] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:06.632879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581ee5d-4559-490d-a858-62b28f0f76ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:06.633917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581ee5d-4559-490d-a858-62b28f0f76ac None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:06.634376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581ee5d-4559-490d-a858-62b28f0f76ac None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:06.634852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581ee5d-4559-490d-a858-62b28f0f76ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:06.711531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581ee5d-4559-490d-a858-62b28f0f76ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:06.714000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581ee5d-4559-490d-a858-62b28f0f76ac None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:06.798870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:06.838676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 210/430] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:06.845048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb5df-2d6c-4ad2-a392-d4fdaaafb3ff None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:06.925493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581ee5d-4559-490d-a858-62b28f0f76ac None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:06.957018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cb5df-2d6c-4ad2-a392-d4fdaaafb3ff None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:06.987881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:07.001078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4ce1d-4dda-442b-98e8-c4686c802a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:07.001765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4ce1d-4dda-442b-98e8-c4686c802a62 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.001980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4ce1d-4dda-442b-98e8-c4686c802a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.002261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4ce1d-4dda-442b-98e8-c4686c802a62 None admin] PATH_INFO: `/v3/projects/811b356a854c422f85cddd17b5c478aa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.003488 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4ce1d-4dda-442b-98e8-c4686c802a62 None admin] RBAC: Authorizing `identity:get_project(project_id=811b356a854c422f85cddd17b5c478aa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:07.005014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4ce1d-4dda-442b-98e8-c4686c802a62 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:07.007184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 221/431] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:06 2022] GET /identity/v3/projects/811b356a854c422f85cddd17b5c478aa => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:07.015418 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4ce1d-4dda-442b-98e8-c4686c802a62 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:07.077617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581ee5d-4559-490d-a858-62b28f0f76ac None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:07.118370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f938f94-30a3-4835-8d14-25c690b3ae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:07.119375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f938f94-30a3-4835-8d14-25c690b3ae72 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.120040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f938f94-30a3-4835-8d14-25c690b3ae72 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.120518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f938f94-30a3-4835-8d14-25c690b3ae72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.132777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f938f94-30a3-4835-8d14-25c690b3ae72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2f938f94-30a3-4835-8d14-25c690b3ae72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:07.138397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:07.139272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 12:07:07.140106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:07.142829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 211/432] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:07.148408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f938f94-30a3-4835-8d14-25c690b3ae72 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:07.173252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4ce1d-4dda-442b-98e8-c4686c802a62 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:07.179357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:07.218509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d788a-97a1-4899-bea6-9924d098df40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:07.220273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d788a-97a1-4899-bea6-9924d098df40 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.220713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d788a-97a1-4899-bea6-9924d098df40 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.221149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d788a-97a1-4899-bea6-9924d098df40 None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.222326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d788a-97a1-4899-bea6-9924d098df40 None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:07.223798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d788a-97a1-4899-bea6-9924d098df40 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:07.225728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 222/433] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:07 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:07.226896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cabcc-bfdd-4a2c-a1fe-6a49700c88a9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.227179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cabcc-bfdd-4a2c-a1fe-6a49700c88a9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.227453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cabcc-bfdd-4a2c-a1fe-6a49700c88a9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.231790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 223/434] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:07.233579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610cabcc-bfdd-4a2c-a1fe-6a49700c88a9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:07.287367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f938f94-30a3-4835-8d14-25c690b3ae72 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:07.303970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:07.338947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62178d0-2195-49a3-ac44-0a5b0abe7c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:07.340014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62178d0-2195-49a3-ac44-0a5b0abe7c96 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.341248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62178d0-2195-49a3-ac44-0a5b0abe7c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.341725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62178d0-2195-49a3-ac44-0a5b0abe7c96 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.342557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62178d0-2195-49a3-ac44-0a5b0abe7c96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:07.343152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62178d0-2195-49a3-ac44-0a5b0abe7c96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:07.344764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62178d0-2195-49a3-ac44-0a5b0abe7c96 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:07.358795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 212/435] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:07:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:07.360106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3735164-8524-492a-8b7a-09ba550eb3eb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.360394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3735164-8524-492a-8b7a-09ba550eb3eb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.360705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3735164-8524-492a-8b7a-09ba550eb3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.364515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610cabcc-bfdd-4a2c-a1fe-6a49700c88a9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:07.384731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b3735164-8524-492a-8b7a-09ba550eb3eb None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:07.386711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b3735164-8524-492a-8b7a-09ba550eb3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:07.407896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99fcfe-9bab-4081-96ec-195138450af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:07.408505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99fcfe-9bab-4081-96ec-195138450af6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.408790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99fcfe-9bab-4081-96ec-195138450af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.409063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99fcfe-9bab-4081-96ec-195138450af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.409792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99fcfe-9bab-4081-96ec-195138450af6 None admin] RBAC: Adding query filter params (group.id=4d5b6821203d4f5689ab71ea92833abb, role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=811b356a854c422f85cddd17b5c478aa) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:07.410107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99fcfe-9bab-4081-96ec-195138450af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:07.411557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99fcfe-9bab-4081-96ec-195138450af6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:07.419877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 224/436] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:07:07 2022] GET /identity/v3/role_assignments?group.id=4d5b6821203d4f5689ab71ea92833abb&scope.project.id=811b356a854c422f85cddd17b5c478aa&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:07.420449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99fcfe-9bab-4081-96ec-195138450af6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:07.480488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 213/437] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:07.557973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99fcfe-9bab-4081-96ec-195138450af6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:07.584007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:07.597896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77523037-00fd-48bf-af60-7fc3b9baa887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:07.598532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77523037-00fd-48bf-af60-7fc3b9baa887 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.598862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77523037-00fd-48bf-af60-7fc3b9baa887 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.599143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77523037-00fd-48bf-af60-7fc3b9baa887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.599605 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b2d06-431f-4324-9097-7e020d893d90 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.599938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b2d06-431f-4324-9097-7e020d893d90 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.600215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b2d06-431f-4324-9097-7e020d893d90 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.601804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 214/438] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:07.615524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c677fc-8226-485b-a0c8-4b41fe23cc3f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.615902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c677fc-8226-485b-a0c8-4b41fe23cc3f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.616188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c677fc-8226-485b-a0c8-4b41fe23cc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.642303 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d6c677fc-8226-485b-a0c8-4b41fe23cc3f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:07.643642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d6c677fc-8226-485b-a0c8-4b41fe23cc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:07.656046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:07.688874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77523037-00fd-48bf-af60-7fc3b9baa887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:07.690342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77523037-00fd-48bf-af60-7fc3b9baa887 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:07.731824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 215/439] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:07.758545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c677fc-8226-485b-a0c8-4b41fe23cc3f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:07.817632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 225/440] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:07.824202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77523037-00fd-48bf-af60-7fc3b9baa887 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:07.856060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:07.884483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c677fc-8226-485b-a0c8-4b41fe23cc3f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:07.929848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb206fd-f9da-4411-8d19-b0f21e7f4368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:07.930480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb206fd-f9da-4411-8d19-b0f21e7f4368 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:07.930756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb206fd-f9da-4411-8d19-b0f21e7f4368 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:07.931162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb206fd-f9da-4411-8d19-b0f21e7f4368 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:07.945786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb206fd-f9da-4411-8d19-b0f21e7f4368 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-8cb206fd-f9da-4411-8d19-b0f21e7f4368 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:07.950903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:07.951821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:07.953253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 216/441] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:07:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:07.959440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb206fd-f9da-4411-8d19-b0f21e7f4368 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:07.965876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:07.987937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77523037-00fd-48bf-af60-7fc3b9baa887 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.033178 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5183634-59a3-49d6-853d-dceedb3e1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.033820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5183634-59a3-49d6-853d-dceedb3e1e97 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.034098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5183634-59a3-49d6-853d-dceedb3e1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.034372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5183634-59a3-49d6-853d-dceedb3e1e97 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.035649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5183634-59a3-49d6-853d-dceedb3e1e97 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:08.037148 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5183634-59a3-49d6-853d-dceedb3e1e97 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:08.038954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 226/442] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:07 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:08.039536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5183634-59a3-49d6-853d-dceedb3e1e97 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.080408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb206fd-f9da-4411-8d19-b0f21e7f4368 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.114338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c495714-7289-4f0f-b1f7-a945c3ff2052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.114931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c495714-7289-4f0f-b1f7-a945c3ff2052 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.115205 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c495714-7289-4f0f-b1f7-a945c3ff2052 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.115479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c495714-7289-4f0f-b1f7-a945c3ff2052 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.116170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c495714-7289-4f0f-b1f7-a945c3ff2052 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:08.116470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c495714-7289-4f0f-b1f7-a945c3ff2052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:08.117852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c495714-7289-4f0f-b1f7-a945c3ff2052 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:08.126246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 217/443] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:08.127043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c495714-7289-4f0f-b1f7-a945c3ff2052 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.134625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:08.145110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5183634-59a3-49d6-853d-dceedb3e1e97 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.180286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23de831-2cd2-4c1a-872e-6d522352f8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.180914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23de831-2cd2-4c1a-872e-6d522352f8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.181192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23de831-2cd2-4c1a-872e-6d522352f8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.181468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23de831-2cd2-4c1a-872e-6d522352f8e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.192110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23de831-2cd2-4c1a-872e-6d522352f8e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:08.196475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c23de831-2cd2-4c1a-872e-6d522352f8e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:08.197175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:08.198387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:08.199225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 227/444] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:08.199905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23de831-2cd2-4c1a-872e-6d522352f8e4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.234936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c495714-7289-4f0f-b1f7-a945c3ff2052 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.267869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f6358-c8cd-42e2-9c82-c119bb3a7d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.268495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f6358-c8cd-42e2-9c82-c119bb3a7d78 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.268776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f6358-c8cd-42e2-9c82-c119bb3a7d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.269049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f6358-c8cd-42e2-9c82-c119bb3a7d78 None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.270159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f6358-c8cd-42e2-9c82-c119bb3a7d78 None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:08.271615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f6358-c8cd-42e2-9c82-c119bb3a7d78 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:08.273226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 218/445] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:08.274001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f6358-c8cd-42e2-9c82-c119bb3a7d78 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.305395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23de831-2cd2-4c1a-872e-6d522352f8e4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.338201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3559d-7dba-4b7c-8c63-8c6076e1a774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.339216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3559d-7dba-4b7c-8c63-8c6076e1a774 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.339490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3559d-7dba-4b7c-8c63-8c6076e1a774 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.339778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3559d-7dba-4b7c-8c63-8c6076e1a774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.359782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:08.375353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f6358-c8cd-42e2-9c82-c119bb3a7d78 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.408051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d74a38-8855-46a9-928c-9cc17d6bef1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.408645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d74a38-8855-46a9-928c-9cc17d6bef1b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.408922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d74a38-8855-46a9-928c-9cc17d6bef1b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.409192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d74a38-8855-46a9-928c-9cc17d6bef1b None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.409783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3559d-7dba-4b7c-8c63-8c6076e1a774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:08.409879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d74a38-8855-46a9-928c-9cc17d6bef1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:08.410126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d74a38-8855-46a9-928c-9cc17d6bef1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:08.411274 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3559d-7dba-4b7c-8c63-8c6076e1a774 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:08.411464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d74a38-8855-46a9-928c-9cc17d6bef1b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:08.412798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:08.420568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 219/446] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:08.421460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d74a38-8855-46a9-928c-9cc17d6bef1b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.512153 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 228/447] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:08.513293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3559d-7dba-4b7c-8c63-8c6076e1a774 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.524250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d74a38-8855-46a9-928c-9cc17d6bef1b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.559129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace06017-f212-4b4c-b26d-0d923f0168f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.559735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace06017-f212-4b4c-b26d-0d923f0168f9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.560049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace06017-f212-4b4c-b26d-0d923f0168f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.560323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace06017-f212-4b4c-b26d-0d923f0168f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.561057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace06017-f212-4b4c-b26d-0d923f0168f9 None admin] RBAC: Adding query filter params (group.id=4d5b6821203d4f5689ab71ea92833abb, role.id=3367be0edc604df081b5ac7a9e2a7b92, scope.project.id=a06126b4287647ca9294ce703dd39691) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:08.561365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace06017-f212-4b4c-b26d-0d923f0168f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:08.562561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:08.562814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace06017-f212-4b4c-b26d-0d923f0168f9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:08.569542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 220/448] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/role_assignments?group.id=4d5b6821203d4f5689ab71ea92833abb&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=3367be0edc604df081b5ac7a9e2a7b92 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:08.570340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace06017-f212-4b4c-b26d-0d923f0168f9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.617936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3559d-7dba-4b7c-8c63-8c6076e1a774 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.646678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:08.658881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb12ff-e290-4642-8826-b37a4016f7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.659507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb12ff-e290-4642-8826-b37a4016f7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.659804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb12ff-e290-4642-8826-b37a4016f7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.660081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb12ff-e290-4642-8826-b37a4016f7a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.702164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace06017-f212-4b4c-b26d-0d923f0168f9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.736219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb12ff-e290-4642-8826-b37a4016f7a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:08.737712 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb12ff-e290-4642-8826-b37a4016f7a9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:08.747431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26396c-ce37-4e13-a49d-f0220d2f8db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.748257 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26396c-ce37-4e13-a49d-f0220d2f8db2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.748434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26396c-ce37-4e13-a49d-f0220d2f8db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.748659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26396c-ce37-4e13-a49d-f0220d2f8db2 None admin] PATH_INFO: `/v3/projects/a06126b4287647ca9294ce703dd39691` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.749831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26396c-ce37-4e13-a49d-f0220d2f8db2 None admin] RBAC: Authorizing `identity:get_project(project_id=a06126b4287647ca9294ce703dd39691)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:08.751509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26396c-ce37-4e13-a49d-f0220d2f8db2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:08.753313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 221/449] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/projects/a06126b4287647ca9294ce703dd39691 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:08.758411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26396c-ce37-4e13-a49d-f0220d2f8db2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.803095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:08.862754 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 229/450] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:08.868564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb12ff-e290-4642-8826-b37a4016f7a9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.891786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26396c-ce37-4e13-a49d-f0220d2f8db2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:08.928946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:08.938150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c8fe6-7b3a-43a6-abbd-e13543c71032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:08.938788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c8fe6-7b3a-43a6-abbd-e13543c71032 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:08.939069 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c8fe6-7b3a-43a6-abbd-e13543c71032 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:08.939339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c8fe6-7b3a-43a6-abbd-e13543c71032 None admin] PATH_INFO: `/v3/groups/4d5b6821203d4f5689ab71ea92833abb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:08.940492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c8fe6-7b3a-43a6-abbd-e13543c71032 None admin] RBAC: Authorizing `identity:get_group(group_id=4d5b6821203d4f5689ab71ea92833abb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:08.942032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c8fe6-7b3a-43a6-abbd-e13543c71032 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:08.943898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 222/451] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/groups/4d5b6821203d4f5689ab71ea92833abb => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:08.950215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c8fe6-7b3a-43a6-abbd-e13543c71032 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:08.989280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb12ff-e290-4642-8826-b37a4016f7a9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:09.023304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfc42a-9136-4114-b3ea-60b9568beb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:09.023963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfc42a-9136-4114-b3ea-60b9568beb79 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:09.024221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfc42a-9136-4114-b3ea-60b9568beb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:09.024489 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfc42a-9136-4114-b3ea-60b9568beb79 None admin] PATH_INFO: `/v3/projects/5d4deb4c3b0e444f92c029c040b5603c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:09.025677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfc42a-9136-4114-b3ea-60b9568beb79 None admin] RBAC: Authorizing `identity:get_project(project_id=5d4deb4c3b0e444f92c029c040b5603c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:09.027687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfc42a-9136-4114-b3ea-60b9568beb79 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:09.029329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 230/452] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/projects/5d4deb4c3b0e444f92c029c040b5603c => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:09.034846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfc42a-9136-4114-b3ea-60b9568beb79 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:09.052613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:09.054317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c8fe6-7b3a-43a6-abbd-e13543c71032 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:09.088365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e1206-c071-475e-b11f-81cd00137126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:09.088968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e1206-c071-475e-b11f-81cd00137126 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:09.089249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e1206-c071-475e-b11f-81cd00137126 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:09.089516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e1206-c071-475e-b11f-81cd00137126 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:09.090230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e1206-c071-475e-b11f-81cd00137126 None admin] RBAC: Adding query filter params (group.id=4d5b6821203d4f5689ab71ea92833abb, role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=a06126b4287647ca9294ce703dd39691) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:09.090537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e1206-c071-475e-b11f-81cd00137126 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:09.092237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e1206-c071-475e-b11f-81cd00137126 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:09.098373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 223/453] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:07:08 2022] GET /identity/v3/role_assignments?group.id=4d5b6821203d4f5689ab71ea92833abb&scope.project.id=a06126b4287647ca9294ce703dd39691&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:09.138811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfc42a-9136-4114-b3ea-60b9568beb79 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:09.175093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec5041-511a-4580-974f-d81a1dc76e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:09.175743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec5041-511a-4580-974f-d81a1dc76e5a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:09.176051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec5041-511a-4580-974f-d81a1dc76e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:09.176348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec5041-511a-4580-974f-d81a1dc76e5a None admin] PATH_INFO: `/v3/groups/9fabaa52b48e4c61868df76149a0a2b4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:09.177523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec5041-511a-4580-974f-d81a1dc76e5a None admin] RBAC: Authorizing `identity:get_group(group_id=9fabaa52b48e4c61868df76149a0a2b4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:09.179007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec5041-511a-4580-974f-d81a1dc76e5a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:09.180677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 231/454] 10.222.0.71 () {56 vars in 1180 bytes} [Wed Sep 21 12:07:09 2022] GET /identity/v3/groups/9fabaa52b48e4c61868df76149a0a2b4 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:09.187211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e1206-c071-475e-b11f-81cd00137126 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:09.249378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:09.293297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e1206-c071-475e-b11f-81cd00137126 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:09.326829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db48d7f4-b4a3-4f7c-9f0c-89c83fcf7038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:09.327438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48d7f4-b4a3-4f7c-9f0c-89c83fcf7038 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:09.327721 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48d7f4-b4a3-4f7c-9f0c-89c83fcf7038 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:09.328025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db48d7f4-b4a3-4f7c-9f0c-89c83fcf7038 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:09.328752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48d7f4-b4a3-4f7c-9f0c-89c83fcf7038 None admin] RBAC: Adding query filter params (group.id=9fabaa52b48e4c61868df76149a0a2b4, role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=5d4deb4c3b0e444f92c029c040b5603c) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:09.329068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48d7f4-b4a3-4f7c-9f0c-89c83fcf7038 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:09.330503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db48d7f4-b4a3-4f7c-9f0c-89c83fcf7038 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:09.336526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 224/455] 10.222.0.71 () {56 vars in 1376 bytes} [Wed Sep 21 12:07:09 2022] GET /identity/v3/role_assignments?group.id=9fabaa52b48e4c61868df76149a0a2b4&scope.project.id=5d4deb4c3b0e444f92c029c040b5603c&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:11.936589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcbf30-ec34-4119-b261-cf43708f1f17 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:11.937118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcbf30-ec34-4119-b261-cf43708f1f17 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:11.937118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcbf30-ec34-4119-b261-cf43708f1f17 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:11.938450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 232/456] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:11.953884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d2dfc-0153-4c8e-b360-f63ea9f37355 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:11.954392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d2dfc-0153-4c8e-b360-f63ea9f37355 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:11.954464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d2dfc-0153-4c8e-b360-f63ea9f37355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:11.976489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5e6d2dfc-0153-4c8e-b360-f63ea9f37355 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:11.977696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5e6d2dfc-0153-4c8e-b360-f63ea9f37355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:12.081588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 225/457] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:12.340336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaf1d0-c87e-427c-aad4-0003e3464c4f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:12.340688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaf1d0-c87e-427c-aad4-0003e3464c4f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:12.340866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eaf1d0-c87e-427c-aad4-0003e3464c4f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:12.342855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 233/458] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:12.349335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d32e1-c8df-457d-ac05-99f760d49e11 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:12.349642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d32e1-c8df-457d-ac05-99f760d49e11 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:12.349886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d32e1-c8df-457d-ac05-99f760d49e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:12.371895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d14d32e1-c8df-457d-ac05-99f760d49e11 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:12.373079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d14d32e1-c8df-457d-ac05-99f760d49e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:12.445319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eaf1d0-c87e-427c-aad4-0003e3464c4f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:12.446795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:12.473582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 226/459] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:12.564872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:12.579548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bdc2a-958b-4f3f-bf0e-d95ae8974abe None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:12.579858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bdc2a-958b-4f3f-bf0e-d95ae8974abe None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:12.580121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bdc2a-958b-4f3f-bf0e-d95ae8974abe None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:12.581454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 227/460] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:12.586686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a86b2-1cfe-4959-aceb-58eb8e31ad5e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:12.587194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a86b2-1cfe-4959-aceb-58eb8e31ad5e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:12.587664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a86b2-1cfe-4959-aceb-58eb8e31ad5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:12.599857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eaf1d0-c87e-427c-aad4-0003e3464c4f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:12.633010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9e2a86b2-1cfe-4959-aceb-58eb8e31ad5e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:12.634220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9e2a86b2-1cfe-4959-aceb-58eb8e31ad5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:12.651403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a28e3-bc39-4afa-a2fa-ad7e2846bd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:12.652066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a28e3-bc39-4afa-a2fa-ad7e2846bd78 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:12.652343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a28e3-bc39-4afa-a2fa-ad7e2846bd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:12.652610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a28e3-bc39-4afa-a2fa-ad7e2846bd78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:12.667858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a28e3-bc39-4afa-a2fa-ad7e2846bd78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:12.670295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a28e3-bc39-4afa-a2fa-ad7e2846bd78 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-329a28e3-bc39-4afa-a2fa-ad7e2846bd78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:12.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:12.700108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:12.700975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:12.700975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:12.700975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:12.701646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 234/461] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:12.706898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a28e3-bc39-4afa-a2fa-ad7e2846bd78 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:12.748670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 228/462] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:07:12.764159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a86b2-1cfe-4959-aceb-58eb8e31ad5e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:12.894730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a28e3-bc39-4afa-a2fa-ad7e2846bd78 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:12.912018 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a86b2-1cfe-4959-aceb-58eb8e31ad5e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:12.927534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:12.933626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960ea99-5ee0-4a3d-bffe-cf9cea417af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:12.934240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960ea99-5ee0-4a3d-bffe-cf9cea417af1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:12.934515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960ea99-5ee0-4a3d-bffe-cf9cea417af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:12.934783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960ea99-5ee0-4a3d-bffe-cf9cea417af1 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:12.935419 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960ea99-5ee0-4a3d-bffe-cf9cea417af1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:12.935717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960ea99-5ee0-4a3d-bffe-cf9cea417af1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:12.937750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960ea99-5ee0-4a3d-bffe-cf9cea417af1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:12.947663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:12.954981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 235/463] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:12.955457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d3403-2bfe-41bf-9682-774cb9c0c507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:12.956361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82911ada-b124-49a9-b345-f65bcc2217e3 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:12.956636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82911ada-b124-49a9-b345-f65bcc2217e3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:12.957091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d3403-2bfe-41bf-9682-774cb9c0c507 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:12.957194 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82911ada-b124-49a9-b345-f65bcc2217e3 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:12.957657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d3403-2bfe-41bf-9682-774cb9c0c507 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:12.958123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d3403-2bfe-41bf-9682-774cb9c0c507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:12.958370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 236/464] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:12.963338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82911ada-b124-49a9-b345-f65bcc2217e3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:12.973808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d3403-2bfe-41bf-9682-774cb9c0c507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:12.975511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d3403-2bfe-41bf-9682-774cb9c0c507 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-476d3403-2bfe-41bf-9682-774cb9c0c507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:12.991501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:12.992638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:12.993668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:12.993668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:12.993668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:12.995550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 229/465] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:13.001163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318030cb-61b1-4efc-abf9-77d740936b26 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.001529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318030cb-61b1-4efc-abf9-77d740936b26 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.001872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318030cb-61b1-4efc-abf9-77d740936b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.021793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-318030cb-61b1-4efc-abf9-77d740936b26 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:13.023284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-318030cb-61b1-4efc-abf9-77d740936b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:13.088609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82911ada-b124-49a9-b345-f65bcc2217e3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:13.107785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 230/466] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:13.107785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318030cb-61b1-4efc-abf9-77d740936b26 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:13.143927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de9afc-e719-4723-94cf-bbf2208b297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:13.144991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9afc-e719-4723-94cf-bbf2208b297d None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.145288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9afc-e719-4723-94cf-bbf2208b297d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.145634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9afc-e719-4723-94cf-bbf2208b297d None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.146362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de9afc-e719-4723-94cf-bbf2208b297d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:13.149182 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de9afc-e719-4723-94cf-bbf2208b297d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:13.177971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 237/467] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:07:12 2022] POST /identity/v3/users => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:13.179220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de9afc-e719-4723-94cf-bbf2208b297d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:13.264589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:13.272427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318030cb-61b1-4efc-abf9-77d740936b26 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:13.308178 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:13.363341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1950a-efd9-4ef6-9e2d-2593d3b00bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:13.363985 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1950a-efd9-4ef6-9e2d-2593d3b00bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.364266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1950a-efd9-4ef6-9e2d-2593d3b00bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.364539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1950a-efd9-4ef6-9e2d-2593d3b00bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.365216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1950a-efd9-4ef6-9e2d-2593d3b00bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:13.365521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1950a-efd9-4ef6-9e2d-2593d3b00bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:13.366948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1950a-efd9-4ef6-9e2d-2593d3b00bd5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:13.388731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 231/468] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:13.389673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1950a-efd9-4ef6-9e2d-2593d3b00bd5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:13.402759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de9afc-e719-4723-94cf-bbf2208b297d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:13.446121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3056ba3-4224-4e86-aa76-b3f7f7a8a26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:13.446937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056ba3-4224-4e86-aa76-b3f7f7a8a26d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.447220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056ba3-4224-4e86-aa76-b3f7f7a8a26d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.448020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056ba3-4224-4e86-aa76-b3f7f7a8a26d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.461651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3056ba3-4224-4e86-aa76-b3f7f7a8a26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:13.464191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3056ba3-4224-4e86-aa76-b3f7f7a8a26d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f3056ba3-4224-4e86-aa76-b3f7f7a8a26d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:13.481042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:13.481979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:13.483226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:13.483226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:13.483226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:13.483226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 238/469] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:13.483226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3056ba3-4224-4e86-aa76-b3f7f7a8a26d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:13.506340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1950a-efd9-4ef6-9e2d-2593d3b00bd5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:13.546808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c70601-7df6-4267-9fc5-2b9b884a0d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:13.547461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c70601-7df6-4267-9fc5-2b9b884a0d47 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.547743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c70601-7df6-4267-9fc5-2b9b884a0d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.548034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c70601-7df6-4267-9fc5-2b9b884a0d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.561429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c70601-7df6-4267-9fc5-2b9b884a0d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:13.562928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c70601-7df6-4267-9fc5-2b9b884a0d47 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-05c70601-7df6-4267-9fc5-2b9b884a0d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:13.578170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:13.579118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:13.580044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:13.580044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:13.580044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:13.580044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 232/470] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:13.580703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d318d4-ea69-438b-b4ff-ec3f18566e95 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.580703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d318d4-ea69-438b-b4ff-ec3f18566e95 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.580989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d318d4-ea69-438b-b4ff-ec3f18566e95 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.582352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 233/471] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:13.582880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d318d4-ea69-438b-b4ff-ec3f18566e95 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:13.595705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:13.600208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3056ba3-4224-4e86-aa76-b3f7f7a8a26d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:13.608872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:13.642298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec1d0-c6f9-46f9-bb7b-ac0125815c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:13.643203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec1d0-c6f9-46f9-bb7b-ac0125815c5e None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.643484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec1d0-c6f9-46f9-bb7b-ac0125815c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.643782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec1d0-c6f9-46f9-bb7b-ac0125815c5e None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.644447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec1d0-c6f9-46f9-bb7b-ac0125815c5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:13.645890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec1d0-c6f9-46f9-bb7b-ac0125815c5e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:13.676052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 239/472] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:07:13 2022] POST /identity/v3/users => generated 248 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:13.678217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec1d0-c6f9-46f9-bb7b-ac0125815c5e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:13.707518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d318d4-ea69-438b-b4ff-ec3f18566e95 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:13.754221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd0742-d673-4893-9d89-59b3197dd1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:13.755513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd0742-d673-4893-9d89-59b3197dd1ab None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.756002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd0742-d673-4893-9d89-59b3197dd1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.756801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd0742-d673-4893-9d89-59b3197dd1ab None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.758245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd0742-d673-4893-9d89-59b3197dd1ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:13.758776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd0742-d673-4893-9d89-59b3197dd1ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:13.761386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd0742-d673-4893-9d89-59b3197dd1ab None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:13.781587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 234/473] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:13.783005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e765e-7efd-4e4e-95ef-865fbbb3fa94 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.783299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e765e-7efd-4e4e-95ef-865fbbb3fa94 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.783575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e765e-7efd-4e4e-95ef-865fbbb3fa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.816634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-849e765e-7efd-4e4e-95ef-865fbbb3fa94 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:13.818370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-849e765e-7efd-4e4e-95ef-865fbbb3fa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:13.830821 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec1d0-c6f9-46f9-bb7b-ac0125815c5e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:13.876553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed675493-9dbb-4e7c-b323-1f0c25cbeae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:13.877050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed675493-9dbb-4e7c-b323-1f0c25cbeae5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:13.877432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed675493-9dbb-4e7c-b323-1f0c25cbeae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:13.877813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed675493-9dbb-4e7c-b323-1f0c25cbeae5 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:13.879508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed675493-9dbb-4e7c-b323-1f0c25cbeae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:13.879646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed675493-9dbb-4e7c-b323-1f0c25cbeae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:13.881172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed675493-9dbb-4e7c-b323-1f0c25cbeae5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:13.897501 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 240/474] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:13.898590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed675493-9dbb-4e7c-b323-1f0c25cbeae5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:13.903082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:13.909168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 235/475] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:13.910102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e765e-7efd-4e4e-95ef-865fbbb3fa94 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:13.930382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:14.019954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed675493-9dbb-4e7c-b323-1f0c25cbeae5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:14.030367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e765e-7efd-4e4e-95ef-865fbbb3fa94 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:14.074550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197627fc-2bdb-4281-9519-dc66f84925c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:14.075262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197627fc-2bdb-4281-9519-dc66f84925c4 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:14.075892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197627fc-2bdb-4281-9519-dc66f84925c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:14.076254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197627fc-2bdb-4281-9519-dc66f84925c4 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:14.077455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197627fc-2bdb-4281-9519-dc66f84925c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:14.079113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197627fc-2bdb-4281-9519-dc66f84925c4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:14.100679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15c210-1f68-401b-bae8-2021583881ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:14.101625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c210-1f68-401b-bae8-2021583881ca None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:14.102978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c210-1f68-401b-bae8-2021583881ca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:14.103496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15c210-1f68-401b-bae8-2021583881ca None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:14.105736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15c210-1f68-401b-bae8-2021583881ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:14.109527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15c210-1f68-401b-bae8-2021583881ca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:14.112874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 236/476] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:07:13 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:14.151493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 241/477] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:07:13 2022] POST /identity/v3/users => generated 248 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:14.240093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197627fc-2bdb-4281-9519-dc66f84925c4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:14.317502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:14.388010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197627fc-2bdb-4281-9519-dc66f84925c4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:14.438289 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:14.438904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:14.439354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:14.439632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:14.440346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:14.441719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:14.457089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.sql.core [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 21 12:07:14.457089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 21 12:07:14.457089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [parameters: {'id': '891e965c0720474ca3d276e1bb793932', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 21 12:07:14.457089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81407) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application conn.query(q) Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application result.read() Sep 21 12:07:14.474310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return ref.to_dict() Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 12:07:14.475356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application next(self.gen) Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application yield resource Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application next(self.gen) Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.session.rollback() Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.force_reraise() Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise self.value Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application session.commit() Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 21 12:07:14.476466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._prepare_impl() Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.session.flush() Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._flush(objects) Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application compat.raise_( Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application flush_context.execute() Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rec.execute(self) Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 21 12:07:14.477668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 12:07:14.478874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application conn.query(q) Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application result.read() Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 12:07:14.480078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [parameters: {'id': '891e965c0720474ca3d276e1bb793932', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:14.481519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 21 12:07:14.486134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:14.488078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 237/478] 10.222.0.71 () {60 vars in 1131 bytes} [Wed Sep 21 12:07:14 2022] POST /identity/v3/roles => generated 144 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:14.488223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15c210-1f68-401b-bae8-2021583881ca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:14.584294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:14.612711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15c210-1f68-401b-bae8-2021583881ca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:14.651276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828c4a9-f229-412d-aa13-7f38e92491cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:14.652360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828c4a9-f229-412d-aa13-7f38e92491cf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:14.652849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828c4a9-f229-412d-aa13-7f38e92491cf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:14.653315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828c4a9-f229-412d-aa13-7f38e92491cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:14.665358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828c4a9-f229-412d-aa13-7f38e92491cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8828c4a9-f229-412d-aa13-7f38e92491cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:14.671786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:14.672656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 12:07:14.673440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:14.676133 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 242/479] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:07:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:14.683810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:14.840309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eeea0-dec3-4ef0-ab53-5653054e4a10 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:14.857648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:14.889950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7e639-d0e8-44ba-8634-9d6fcea10163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:14.890597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7e639-d0e8-44ba-8634-9d6fcea10163 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:14.890876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7e639-d0e8-44ba-8634-9d6fcea10163 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:14.891148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7e639-d0e8-44ba-8634-9d6fcea10163 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:14.891832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7e639-d0e8-44ba-8634-9d6fcea10163 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:14.892135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7e639-d0e8-44ba-8634-9d6fcea10163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:14.893546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7e639-d0e8-44ba-8634-9d6fcea10163 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:14.903455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 238/480] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:15.724896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa987e37-b67d-4f0c-a3c5-06c2009c7a26 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:15.725956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa987e37-b67d-4f0c-a3c5-06c2009c7a26 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:15.726460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa987e37-b67d-4f0c-a3c5-06c2009c7a26 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:15.728566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 243/481] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:15.735080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cd7e7-0b9b-4e3c-979c-de3648e25af3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:15.736488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cd7e7-0b9b-4e3c-979c-de3648e25af3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:15.737293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cd7e7-0b9b-4e3c-979c-de3648e25af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:15.772901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-163cd7e7-0b9b-4e3c-979c-de3648e25af3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:15.774107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-163cd7e7-0b9b-4e3c-979c-de3648e25af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:15.861402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 239/482] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:16.164212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa987e37-b67d-4f0c-a3c5-06c2009c7a26 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:16.293046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa987e37-b67d-4f0c-a3c5-06c2009c7a26 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:16.338716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c5ede-09c9-4989-b969-9e39a2cfd6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:16.339391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c5ede-09c9-4989-b969-9e39a2cfd6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:16.339673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c5ede-09c9-4989-b969-9e39a2cfd6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:16.339973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c5ede-09c9-4989-b969-9e39a2cfd6f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:16.353861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:16.363160 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c5ede-09c9-4989-b969-9e39a2cfd6f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-107c5ede-09c9-4989-b969-9e39a2cfd6f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:16.368342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:16.369366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:16.370161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 244/483] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:16.377701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cd7e7-0b9b-4e3c-979c-de3648e25af3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:16.496968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab67cfb-67e4-4177-99c5-f982203c6233 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:16.497282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab67cfb-67e4-4177-99c5-f982203c6233 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:16.497446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab67cfb-67e4-4177-99c5-f982203c6233 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:16.499031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 245/484] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:16.509234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58a248-7a7f-4458-b2dd-3e8ec28d2f92 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:16.509520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58a248-7a7f-4458-b2dd-3e8ec28d2f92 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:16.509794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58a248-7a7f-4458-b2dd-3e8ec28d2f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:16.538780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cd7e7-0b9b-4e3c-979c-de3648e25af3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:16.545403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0b58a248-7a7f-4458-b2dd-3e8ec28d2f92 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:16.546886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0b58a248-7a7f-4458-b2dd-3e8ec28d2f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:16.552257 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:16.621586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c9aaa-d3f1-4ca2-9277-d168478565e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:16.622227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c9aaa-d3f1-4ca2-9277-d168478565e4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:16.622510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c9aaa-d3f1-4ca2-9277-d168478565e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:16.622779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c9aaa-d3f1-4ca2-9277-d168478565e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:16.623446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c9aaa-d3f1-4ca2-9277-d168478565e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:16.623744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c9aaa-d3f1-4ca2-9277-d168478565e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:16.624804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 246/485] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:16.625895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c9aaa-d3f1-4ca2-9277-d168478565e4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:16.636228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 240/486] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:16.642211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58a248-7a7f-4458-b2dd-3e8ec28d2f92 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:16.751599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58a248-7a7f-4458-b2dd-3e8ec28d2f92 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:16.781151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:16.790793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032007-5b74-4962-a22d-9d8e9125b690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:16.791570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032007-5b74-4962-a22d-9d8e9125b690 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:16.791879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032007-5b74-4962-a22d-9d8e9125b690 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:16.792154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032007-5b74-4962-a22d-9d8e9125b690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:16.803219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0032007-5b74-4962-a22d-9d8e9125b690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:16.804718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0032007-5b74-4962-a22d-9d8e9125b690 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b0032007-5b74-4962-a22d-9d8e9125b690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:16.820660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:16.821556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:16.823345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:16.823345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:16.823345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:16.823345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 247/487] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:16.828209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c9aaa-d3f1-4ca2-9277-d168478565e4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:16.958215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c9aaa-d3f1-4ca2-9277-d168478565e4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:17.013561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c770268-8e8a-4fdd-bd5a-013e4e3fb4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:17.014475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c770268-8e8a-4fdd-bd5a-013e4e3fb4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.014930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c770268-8e8a-4fdd-bd5a-013e4e3fb4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.015390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c770268-8e8a-4fdd-bd5a-013e4e3fb4b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.016234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c770268-8e8a-4fdd-bd5a-013e4e3fb4b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:17.016702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c770268-8e8a-4fdd-bd5a-013e4e3fb4b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:17.018250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c770268-8e8a-4fdd-bd5a-013e4e3fb4b1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:17.038105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 241/488] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:17.044132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032007-5b74-4962-a22d-9d8e9125b690 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:17.053194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c770268-8e8a-4fdd-bd5a-013e4e3fb4b1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:17.108515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:17.181097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032007-5b74-4962-a22d-9d8e9125b690 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:17.181982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c770268-8e8a-4fdd-bd5a-013e4e3fb4b1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:17.217127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f255b-e600-4121-9dcc-64d7bcb865b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:17.217529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee840af-2003-42e5-bb3f-c3799c2e4954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:17.217750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f255b-e600-4121-9dcc-64d7bcb865b5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.218026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f255b-e600-4121-9dcc-64d7bcb865b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.218184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee840af-2003-42e5-bb3f-c3799c2e4954 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.218284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f255b-e600-4121-9dcc-64d7bcb865b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.218436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee840af-2003-42e5-bb3f-c3799c2e4954 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.218699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee840af-2003-42e5-bb3f-c3799c2e4954 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.226295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f255b-e600-4121-9dcc-64d7bcb865b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:17.227945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f255b-e600-4121-9dcc-64d7bcb865b5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:17.229376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee840af-2003-42e5-bb3f-c3799c2e4954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-eee840af-2003-42e5-bb3f-c3799c2e4954 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:17.234225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:17.235075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:17.235881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 242/489] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:17.237033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e99001-43b6-406f-a304-25d271160e4e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.237144 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e99001-43b6-406f-a304-25d271160e4e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.237409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e99001-43b6-406f-a304-25d271160e4e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.238810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 243/490] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:17.243904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a306b-7c1b-4bb0-a25d-d18b2e7206f6 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.244201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a306b-7c1b-4bb0-a25d-d18b2e7206f6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.244535 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a306b-7c1b-4bb0-a25d-d18b2e7206f6 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0a7f255b-e600-4121-9dcc-64d7bcb865b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:17.245888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:17.253521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:17.255965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:17.257215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 244/491] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:17.257352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 248/492] 10.222.0.71 () {56 vars in 1093 bytes} [Wed Sep 21 12:07:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:17.257483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cd89b-4630-4cb7-8be5-8aa6ae538f9b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.257483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cd89b-4630-4cb7-8be5-8aa6ae538f9b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.257483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cd89b-4630-4cb7-8be5-8aa6ae538f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.257871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f255b-e600-4121-9dcc-64d7bcb865b5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:17.272053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-461cd89b-4630-4cb7-8be5-8aa6ae538f9b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:17.273178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-461cd89b-4630-4cb7-8be5-8aa6ae538f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:17.300039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:17.355786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 245/493] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:17.355786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7dd84-e613-4c98-9144-31bc4b521283 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.355786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7dd84-e613-4c98-9144-31bc4b521283 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.355786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7dd84-e613-4c98-9144-31bc4b521283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.373021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9bd7dd84-e613-4c98-9144-31bc4b521283 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:17.374354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9bd7dd84-e613-4c98-9144-31bc4b521283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:17.383715 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f255b-e600-4121-9dcc-64d7bcb865b5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:17.392604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:17.433603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059aaee-d6af-4014-8d7a-c8cb67c1a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:17.434909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059aaee-d6af-4014-8d7a-c8cb67c1a8cd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.435265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059aaee-d6af-4014-8d7a-c8cb67c1a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.435614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059aaee-d6af-4014-8d7a-c8cb67c1a8cd None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.436671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059aaee-d6af-4014-8d7a-c8cb67c1a8cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:17.437048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059aaee-d6af-4014-8d7a-c8cb67c1a8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:17.439543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059aaee-d6af-4014-8d7a-c8cb67c1a8cd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:17.450526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 246/494] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:17.451139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 249/495] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:17.456017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7dd84-e613-4c98-9144-31bc4b521283 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:17.457647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059aaee-d6af-4014-8d7a-c8cb67c1a8cd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:17.576854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059aaee-d6af-4014-8d7a-c8cb67c1a8cd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:17.578636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:17.616264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49505581-4166-468d-8bc8-5910a4e89bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:17.617276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49505581-4166-468d-8bc8-5910a4e89bcf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.617727 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49505581-4166-468d-8bc8-5910a4e89bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.618192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49505581-4166-468d-8bc8-5910a4e89bcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.631497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49505581-4166-468d-8bc8-5910a4e89bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:17.633477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49505581-4166-468d-8bc8-5910a4e89bcf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-49505581-4166-468d-8bc8-5910a4e89bcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:17.650847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:17.651799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:17.652625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:17.652625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:17.652625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:17.659460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 250/496] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:17.661482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49505581-4166-468d-8bc8-5910a4e89bcf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:17.664012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7dd84-e613-4c98-9144-31bc4b521283 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:17.733651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc59c2-614c-4608-8204-8059ca15deed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:17.734709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc59c2-614c-4608-8204-8059ca15deed None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.735198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc59c2-614c-4608-8204-8059ca15deed None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.735637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc59c2-614c-4608-8204-8059ca15deed None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.736608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc59c2-614c-4608-8204-8059ca15deed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:17.737105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc59c2-614c-4608-8204-8059ca15deed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:17.738947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc59c2-614c-4608-8204-8059ca15deed None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:17.790769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49505581-4166-468d-8bc8-5910a4e89bcf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:17.794460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 247/497] 10.222.0.71 () {56 vars in 1133 bytes} [Wed Sep 21 12:07:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:17.795631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc59c2-614c-4608-8204-8059ca15deed None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:17.808890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:17.826275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab9a57-4bdf-4546-9e27-142b6ef4e696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:17.826948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab9a57-4bdf-4546-9e27-142b6ef4e696 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.827157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab9a57-4bdf-4546-9e27-142b6ef4e696 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.827453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab9a57-4bdf-4546-9e27-142b6ef4e696 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.828102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab9a57-4bdf-4546-9e27-142b6ef4e696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:17.828401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab9a57-4bdf-4546-9e27-142b6ef4e696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:17.829777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab9a57-4bdf-4546-9e27-142b6ef4e696 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:17.843044 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 251/498] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:17.843927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab9a57-4bdf-4546-9e27-142b6ef4e696 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:17.859535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:17.905404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc59c2-614c-4608-8204-8059ca15deed None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:17.939331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4906980-aeb3-4ea4-812a-3184cae11571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:17.939996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4906980-aeb3-4ea4-812a-3184cae11571 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.940278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4906980-aeb3-4ea4-812a-3184cae11571 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.940551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4906980-aeb3-4ea4-812a-3184cae11571 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:17.946997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab9a57-4bdf-4546-9e27-142b6ef4e696 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:17.952541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4906980-aeb3-4ea4-812a-3184cae11571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c4906980-aeb3-4ea4-812a-3184cae11571 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:17.957437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:17.958322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:17.959124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 248/499] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:17.959808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4906980-aeb3-4ea4-812a-3184cae11571 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:17.984931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fa073-db26-47e7-9baa-97f4791b460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:17.985530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fa073-db26-47e7-9baa-97f4791b460a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:17.985808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fa073-db26-47e7-9baa-97f4791b460a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:17.986078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fa073-db26-47e7-9baa-97f4791b460a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.028232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:18.054800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fa073-db26-47e7-9baa-97f4791b460a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:18.056389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fa073-db26-47e7-9baa-97f4791b460a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:18.071112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4906980-aeb3-4ea4-812a-3184cae11571 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:18.071536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:18.124221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553c34f-5312-499e-b91e-db2cd8e8dd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:18.125020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553c34f-5312-499e-b91e-db2cd8e8dd43 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.125387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553c34f-5312-499e-b91e-db2cd8e8dd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.125840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553c34f-5312-499e-b91e-db2cd8e8dd43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.142256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553c34f-5312-499e-b91e-db2cd8e8dd43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-7553c34f-5312-499e-b91e-db2cd8e8dd43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:18.150913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:18.151950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:18.152759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 249/500] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:18.154317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553c34f-5312-499e-b91e-db2cd8e8dd43 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:18.185740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 252/501] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:18.187011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf6694-e2bd-4b12-97db-5e0a09557f5a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.187306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf6694-e2bd-4b12-97db-5e0a09557f5a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.187614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf6694-e2bd-4b12-97db-5e0a09557f5a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.193045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 253/502] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:18.193726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf6694-e2bd-4b12-97db-5e0a09557f5a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:18.257593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553c34f-5312-499e-b91e-db2cd8e8dd43 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:18.286704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:18.293955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe91b-8712-4b82-bf26-c9a85797f449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:18.294360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf6694-e2bd-4b12-97db-5e0a09557f5a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:18.294585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe91b-8712-4b82-bf26-c9a85797f449 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.294865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe91b-8712-4b82-bf26-c9a85797f449 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.295138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe91b-8712-4b82-bf26-c9a85797f449 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.302433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfe91b-8712-4b82-bf26-c9a85797f449 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:18.304013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfe91b-8712-4b82-bf26-c9a85797f449 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:18.310405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-94cfe91b-8712-4b82-bf26-c9a85797f449 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:18.317284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:18.318194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:18.319247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 250/503] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:18.320320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe91b-8712-4b82-bf26-c9a85797f449 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:18.335798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c1c95-b8ca-4099-9f08-e8f81216c3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:18.336412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c1c95-b8ca-4099-9f08-e8f81216c3df None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.336692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c1c95-b8ca-4099-9f08-e8f81216c3df None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.336962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c1c95-b8ca-4099-9f08-e8f81216c3df None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.337626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c1c95-b8ca-4099-9f08-e8f81216c3df None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:18.337934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c1c95-b8ca-4099-9f08-e8f81216c3df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:18.339337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c1c95-b8ca-4099-9f08-e8f81216c3df None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:18.347512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 254/504] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:18.348332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c1c95-b8ca-4099-9f08-e8f81216c3df None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:18.429511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe91b-8712-4b82-bf26-c9a85797f449 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:18.462659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c1c95-b8ca-4099-9f08-e8f81216c3df None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:18.467407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b365b50-72ce-4658-b19e-323d212e2d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:18.468050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b365b50-72ce-4658-b19e-323d212e2d0f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.468331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b365b50-72ce-4658-b19e-323d212e2d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.468660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b365b50-72ce-4658-b19e-323d212e2d0f None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.469263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b365b50-72ce-4658-b19e-323d212e2d0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:18.469571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b365b50-72ce-4658-b19e-323d212e2d0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:18.470971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b365b50-72ce-4658-b19e-323d212e2d0f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:18.479983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 251/505] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:18.480983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781c820-7837-493f-879c-3ca69d512d5c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.481468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781c820-7837-493f-879c-3ca69d512d5c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.481571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781c820-7837-493f-879c-3ca69d512d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.499967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6781c820-7837-493f-879c-3ca69d512d5c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:18.501432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6781c820-7837-493f-879c-3ca69d512d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:18.501595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138f58a-fb31-4ffa-b247-7debe929a065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:18.502223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138f58a-fb31-4ffa-b247-7debe929a065 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.502507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138f58a-fb31-4ffa-b247-7debe929a065 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.502777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138f58a-fb31-4ffa-b247-7debe929a065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.513514 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138f58a-fb31-4ffa-b247-7debe929a065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:18.514967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138f58a-fb31-4ffa-b247-7debe929a065 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c138f58a-fb31-4ffa-b247-7debe929a065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:18.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:18.529810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:18.530651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:18.530651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:18.530651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:18.530651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 255/506] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:18.530870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138f58a-fb31-4ffa-b247-7debe929a065 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:18.553702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:18.560311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:18.574390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 252/507] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:18.575219 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781c820-7837-493f-879c-3ca69d512d5c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:18.641780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138f58a-fb31-4ffa-b247-7debe929a065 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:18.677712 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d0b2f-51d4-47d7-a14b-ab59f954e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:18.678308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d0b2f-51d4-47d7-a14b-ab59f954e28b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.678688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d0b2f-51d4-47d7-a14b-ab59f954e28b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.678955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d0b2f-51d4-47d7-a14b-ab59f954e28b None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.679913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d0b2f-51d4-47d7-a14b-ab59f954e28b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:18.680223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d0b2f-51d4-47d7-a14b-ab59f954e28b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:18.681705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d0b2f-51d4-47d7-a14b-ab59f954e28b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:18.688297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781c820-7837-493f-879c-3ca69d512d5c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:18.724413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6175d-8dfe-4e81-bbf0-0d98c6692b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:18.725044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6175d-8dfe-4e81-bbf0-0d98c6692b5b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.725587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6175d-8dfe-4e81-bbf0-0d98c6692b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.725693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 256/508] 10.222.0.71 () {56 vars in 1137 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:18.725770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6175d-8dfe-4e81-bbf0-0d98c6692b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.726275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d0b2f-51d4-47d7-a14b-ab59f954e28b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:18.749423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6175d-8dfe-4e81-bbf0-0d98c6692b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:18.751927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6175d-8dfe-4e81-bbf0-0d98c6692b5b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-4cb6175d-8dfe-4e81-bbf0-0d98c6692b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:18.772679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:18.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:18.774423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:18.774423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:18.774423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:18.774423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 253/509] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:18.775116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6175d-8dfe-4e81-bbf0-0d98c6692b5b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:18.810501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:18.831454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:18.849599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d0b2f-51d4-47d7-a14b-ab59f954e28b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:18.879285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6175d-8dfe-4e81-bbf0-0d98c6692b5b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:18.883246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a8dc2-ecdd-4188-a6e1-0b218d089c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:18.883901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a8dc2-ecdd-4188-a6e1-0b218d089c83 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.884181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a8dc2-ecdd-4188-a6e1-0b218d089c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.884454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a8dc2-ecdd-4188-a6e1-0b218d089c83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.895266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a8dc2-ecdd-4188-a6e1-0b218d089c83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:18.896824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a8dc2-ecdd-4188-a6e1-0b218d089c83 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7a0a8dc2-ecdd-4188-a6e1-0b218d089c83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:18.911243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:18.912331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:18.913617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:18.913617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:18.913617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:18.913617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 257/510] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:18.913617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a8dc2-ecdd-4188-a6e1-0b218d089c83 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:18.918420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23308-66f7-4b72-b727-ea6b7b1a1587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:18.919029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23308-66f7-4b72-b727-ea6b7b1a1587 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:18.919308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23308-66f7-4b72-b727-ea6b7b1a1587 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:18.919578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c23308-66f7-4b72-b727-ea6b7b1a1587 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:18.920250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23308-66f7-4b72-b727-ea6b7b1a1587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:18.920553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23308-66f7-4b72-b727-ea6b7b1a1587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:18.921945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c23308-66f7-4b72-b727-ea6b7b1a1587 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:18.935124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 254/511] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:18.935902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23308-66f7-4b72-b727-ea6b7b1a1587 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.021621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a8dc2-ecdd-4188-a6e1-0b218d089c83 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.028717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.035963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c23308-66f7-4b72-b727-ea6b7b1a1587 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.054806 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb031e-63f4-45f9-9427-93a3a9b98612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.055408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb031e-63f4-45f9-9427-93a3a9b98612 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.055691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb031e-63f4-45f9-9427-93a3a9b98612 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.056004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb031e-63f4-45f9-9427-93a3a9b98612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.065100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.069952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f811a-f941-48e4-ac2f-ab3f1c90f432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.070557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f811a-f941-48e4-ac2f-ab3f1c90f432 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.070831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f811a-f941-48e4-ac2f-ab3f1c90f432 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.071098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f811a-f941-48e4-ac2f-ab3f1c90f432 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.071723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f811a-f941-48e4-ac2f-ab3f1c90f432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:19.072046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f811a-f941-48e4-ac2f-ab3f1c90f432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.073980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f811a-f941-48e4-ac2f-ab3f1c90f432 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.085041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 255/512] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:19.085830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f811a-f941-48e4-ac2f-ab3f1c90f432 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.133302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb031e-63f4-45f9-9427-93a3a9b98612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.134791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb031e-63f4-45f9-9427-93a3a9b98612 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.184802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f811a-f941-48e4-ac2f-ab3f1c90f432 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.217806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77205081-6cfc-4676-8f0a-b2678d878873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.218355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77205081-6cfc-4676-8f0a-b2678d878873 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.218627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77205081-6cfc-4676-8f0a-b2678d878873 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.218891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77205081-6cfc-4676-8f0a-b2678d878873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.230032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77205081-6cfc-4676-8f0a-b2678d878873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.231467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77205081-6cfc-4676-8f0a-b2678d878873 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.233613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 258/513] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:19.234474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb031e-63f4-45f9-9427-93a3a9b98612 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-77205081-6cfc-4676-8f0a-b2678d878873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:19.246458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:19.247551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:19.248432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:19.248432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:19.248432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:19.250036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 256/514] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:19.250036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77205081-6cfc-4676-8f0a-b2678d878873 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.258448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.277366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.337508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb031e-63f4-45f9-9427-93a3a9b98612 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.348990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77205081-6cfc-4676-8f0a-b2678d878873 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.372311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a187ff1-55dd-404f-9078-6dac412f34ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.372913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a187ff1-55dd-404f-9078-6dac412f34ea None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.373187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a187ff1-55dd-404f-9078-6dac412f34ea None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.373463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a187ff1-55dd-404f-9078-6dac412f34ea None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.374093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a187ff1-55dd-404f-9078-6dac412f34ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:19.374416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a187ff1-55dd-404f-9078-6dac412f34ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.375791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a187ff1-55dd-404f-9078-6dac412f34ea None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.383743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884017ae-1fc4-46ad-9677-d758efbf2a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.384401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884017ae-1fc4-46ad-9677-d758efbf2a08 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.384683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884017ae-1fc4-46ad-9677-d758efbf2a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.384983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884017ae-1fc4-46ad-9677-d758efbf2a08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.386811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 259/515] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:19.387613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a187ff1-55dd-404f-9078-6dac412f34ea None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.396079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884017ae-1fc4-46ad-9677-d758efbf2a08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.397497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884017ae-1fc4-46ad-9677-d758efbf2a08 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-884017ae-1fc4-46ad-9677-d758efbf2a08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:19.409262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:19.410270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:19.411130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:19.411130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:19.411130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:19.411130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:19.411130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:19.411130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:19.411130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:19.411130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:19.415142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 257/516] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:19.415920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884017ae-1fc4-46ad-9677-d758efbf2a08 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.481067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.483540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.488292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a187ff1-55dd-404f-9078-6dac412f34ea None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.516715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884017ae-1fc4-46ad-9677-d758efbf2a08 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.525983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7c03a-4e59-48e7-afcb-e38f1eeb810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.525983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7c03a-4e59-48e7-afcb-e38f1eeb810a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.525983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7c03a-4e59-48e7-afcb-e38f1eeb810a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.525983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7c03a-4e59-48e7-afcb-e38f1eeb810a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.532533 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7c03a-4e59-48e7-afcb-e38f1eeb810a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.534014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7c03a-4e59-48e7-afcb-e38f1eeb810a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d1d7c03a-4e59-48e7-afcb-e38f1eeb810a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:19.546432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:19.547378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 260/517] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:19.548700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7c03a-4e59-48e7-afcb-e38f1eeb810a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.556155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f636f84-a243-4c19-93d8-610b5771d515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.556821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f636f84-a243-4c19-93d8-610b5771d515 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.557103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f636f84-a243-4c19-93d8-610b5771d515 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.557372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f636f84-a243-4c19-93d8-610b5771d515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.568111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f636f84-a243-4c19-93d8-610b5771d515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.569567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f636f84-a243-4c19-93d8-610b5771d515 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-8f636f84-a243-4c19-93d8-610b5771d515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:19.583776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:19.584859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:19.585710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:19.585710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:19.585710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:19.585710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 258/518] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:19.585710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f636f84-a243-4c19-93d8-610b5771d515 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.648941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7c03a-4e59-48e7-afcb-e38f1eeb810a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.682933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f333e4-5251-401b-812c-9e774f27263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.683540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f333e4-5251-401b-812c-9e774f27263b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.683855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f333e4-5251-401b-812c-9e774f27263b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.684136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f333e4-5251-401b-812c-9e774f27263b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.684780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f333e4-5251-401b-812c-9e774f27263b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:19.685085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f333e4-5251-401b-812c-9e774f27263b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.686473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f333e4-5251-401b-812c-9e774f27263b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.686994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f636f84-a243-4c19-93d8-610b5771d515 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.702590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 261/519] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:19.702590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f333e4-5251-401b-812c-9e774f27263b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.704234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.719323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.723094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc09abb-b5b1-47ed-8ee3-dee6ee8390c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.723746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc09abb-b5b1-47ed-8ee3-dee6ee8390c0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.724051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc09abb-b5b1-47ed-8ee3-dee6ee8390c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.724332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc09abb-b5b1-47ed-8ee3-dee6ee8390c0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.732489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc09abb-b5b1-47ed-8ee3-dee6ee8390c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.733881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc09abb-b5b1-47ed-8ee3-dee6ee8390c0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-7bc09abb-b5b1-47ed-8ee3-dee6ee8390c0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:19.745159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:19.746255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 259/520] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:19.747368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc09abb-b5b1-47ed-8ee3-dee6ee8390c0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.810239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f333e4-5251-401b-812c-9e774f27263b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.844268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a74fb7-773d-460f-8c1c-244f3416c829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.844876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a74fb7-773d-460f-8c1c-244f3416c829 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.845135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a74fb7-773d-460f-8c1c-244f3416c829 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.845409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a74fb7-773d-460f-8c1c-244f3416c829 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.845688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc09abb-b5b1-47ed-8ee3-dee6ee8390c0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:19.846102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a74fb7-773d-460f-8c1c-244f3416c829 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:19.846407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a74fb7-773d-460f-8c1c-244f3416c829 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.847866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a74fb7-773d-460f-8c1c-244f3416c829 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.865497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 262/521] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:19.866308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a74fb7-773d-460f-8c1c-244f3416c829 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.881078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bd9c5-b9f7-4813-8ffb-46c7436105f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:19.881686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bd9c5-b9f7-4813-8ffb-46c7436105f2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:19.881968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bd9c5-b9f7-4813-8ffb-46c7436105f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:19.882240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bd9c5-b9f7-4813-8ffb-46c7436105f2 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:19.882963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bd9c5-b9f7-4813-8ffb-46c7436105f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:19.883269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bd9c5-b9f7-4813-8ffb-46c7436105f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:19.884721 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bd9c5-b9f7-4813-8ffb-46c7436105f2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:19.903146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 260/522] 10.222.0.71 () {56 vars in 1139 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:19.903999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bd9c5-b9f7-4813-8ffb-46c7436105f2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:19.938620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.946584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:19.969214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a74fb7-773d-460f-8c1c-244f3416c829 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.003454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a68c5-d5e2-4b1f-acce-29c088fdc2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:20.003707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bd9c5-b9f7-4813-8ffb-46c7436105f2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.004009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a68c5-d5e2-4b1f-acce-29c088fdc2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:20.004289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a68c5-d5e2-4b1f-acce-29c088fdc2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:20.004561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a68c5-d5e2-4b1f-acce-29c088fdc2e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:20.005199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a68c5-d5e2-4b1f-acce-29c088fdc2e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:20.005502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a68c5-d5e2-4b1f-acce-29c088fdc2e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:20.006871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a68c5-d5e2-4b1f-acce-29c088fdc2e3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:20.018526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 263/523] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:20.019315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a68c5-d5e2-4b1f-acce-29c088fdc2e3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:20.042935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a56fd-1458-4769-bf50-abccc62b65e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:20.043539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a56fd-1458-4769-bf50-abccc62b65e5 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:20.043879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a56fd-1458-4769-bf50-abccc62b65e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:20.044127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a56fd-1458-4769-bf50-abccc62b65e5 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:20.044859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a56fd-1458-4769-bf50-abccc62b65e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:20.046183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a56fd-1458-4769-bf50-abccc62b65e5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:20.066510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 261/524] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:07:19 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:20.067317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a56fd-1458-4769-bf50-abccc62b65e5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:20.155298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a68c5-d5e2-4b1f-acce-29c088fdc2e3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.170197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:20.182690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a56fd-1458-4769-bf50-abccc62b65e5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.195633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1e351-a18e-4e65-854f-b7facc46a0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:20.196700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1e351-a18e-4e65-854f-b7facc46a0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:20.197131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1e351-a18e-4e65-854f-b7facc46a0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:20.198221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1e351-a18e-4e65-854f-b7facc46a0d2 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:20.198991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1e351-a18e-4e65-854f-b7facc46a0d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:20.199306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1e351-a18e-4e65-854f-b7facc46a0d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:20.200966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1e351-a18e-4e65-854f-b7facc46a0d2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:20.211051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:20.223237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 264/525] 10.222.0.71 () {56 vars in 1137 bytes} [Wed Sep 21 12:07:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:20.224383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1e351-a18e-4e65-854f-b7facc46a0d2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:20.230363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873388e-1958-4126-8c51-cdfe8e270d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:20.231105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873388e-1958-4126-8c51-cdfe8e270d3f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:20.231391 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873388e-1958-4126-8c51-cdfe8e270d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:20.231800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873388e-1958-4126-8c51-cdfe8e270d3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:20.232557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873388e-1958-4126-8c51-cdfe8e270d3f None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=463487449c1d4aada1b5f279c959b0e6) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:20.232866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873388e-1958-4126-8c51-cdfe8e270d3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:20.234337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873388e-1958-4126-8c51-cdfe8e270d3f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:20.241231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 262/526] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:20 2022] GET /identity/v3/role_assignments?user.id=463487449c1d4aada1b5f279c959b0e6&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:20.241878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873388e-1958-4126-8c51-cdfe8e270d3f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:20.379289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1e351-a18e-4e65-854f-b7facc46a0d2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.384011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873388e-1958-4126-8c51-cdfe8e270d3f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.428419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e454286-a145-4d0f-91f6-f2a833cf10c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:20.428623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359b3a-d168-4736-83fd-36bf3194dc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:20.429182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e454286-a145-4d0f-91f6-f2a833cf10c8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:20.429539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e454286-a145-4d0f-91f6-f2a833cf10c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:20.429844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359b3a-d168-4736-83fd-36bf3194dc83 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:20.429944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e454286-a145-4d0f-91f6-f2a833cf10c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:20.430382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359b3a-d168-4736-83fd-36bf3194dc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:20.430976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02359b3a-d168-4736-83fd-36bf3194dc83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:20.438247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e454286-a145-4d0f-91f6-f2a833cf10c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:20.439881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e454286-a145-4d0f-91f6-f2a833cf10c8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5e454286-a145-4d0f-91f6-f2a833cf10c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:20.454315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:20.455297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:20.456162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:20.456162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:20.456162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:20.456162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:20.456162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:20.456162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:20.456162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:20.456162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:20.456162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 263/527] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:20.456798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e454286-a145-4d0f-91f6-f2a833cf10c8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:20.478236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:20.513362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02359b3a-d168-4736-83fd-36bf3194dc83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:20.515343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02359b3a-d168-4736-83fd-36bf3194dc83 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:20.520329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:20.575690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e454286-a145-4d0f-91f6-f2a833cf10c8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.614552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e739dc4-18e5-4d90-b00e-013b60256a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:20.615621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e739dc4-18e5-4d90-b00e-013b60256a4a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:20.616007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e739dc4-18e5-4d90-b00e-013b60256a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:20.616621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e739dc4-18e5-4d90-b00e-013b60256a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:20.626254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 265/528] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:20.627484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359b3a-d168-4736-83fd-36bf3194dc83 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:20.694957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e739dc4-18e5-4d90-b00e-013b60256a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:20.696775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e739dc4-18e5-4d90-b00e-013b60256a4a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:20.714800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:20.770201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:20.778956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02359b3a-d168-4736-83fd-36bf3194dc83 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.815066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3eb0d-34dc-4bf6-8b92-02ccfd02c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:20.816103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3eb0d-34dc-4bf6-8b92-02ccfd02c995 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:20.816580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3eb0d-34dc-4bf6-8b92-02ccfd02c995 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:20.817077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3eb0d-34dc-4bf6-8b92-02ccfd02c995 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:20.817988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3eb0d-34dc-4bf6-8b92-02ccfd02c995 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:20.818484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3eb0d-34dc-4bf6-8b92-02ccfd02c995 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:20.820127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3eb0d-34dc-4bf6-8b92-02ccfd02c995 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:20.829339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 264/529] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:20.832082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e739dc4-18e5-4d90-b00e-013b60256a4a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:20.839685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 266/530] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:20.844673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3eb0d-34dc-4bf6-8b92-02ccfd02c995 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:20.943610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e739dc4-18e5-4d90-b00e-013b60256a4a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.967291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:20.981863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3eb0d-34dc-4bf6-8b92-02ccfd02c995 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:20.991371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074fc1-cdbb-44e5-b902-a37834858a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:20.992039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074fc1-cdbb-44e5-b902-a37834858a0e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:20.992316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074fc1-cdbb-44e5-b902-a37834858a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:20.992585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074fc1-cdbb-44e5-b902-a37834858a0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:21.006458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc074fc1-cdbb-44e5-b902-a37834858a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:21.008196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc074fc1-cdbb-44e5-b902-a37834858a0e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:21.021754 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ae1c6-2097-483c-a425-24ed92ad014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:21.022677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ae1c6-2097-483c-a425-24ed92ad014a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:21.023147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ae1c6-2097-483c-a425-24ed92ad014a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:21.023610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ae1c6-2097-483c-a425-24ed92ad014a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-dc074fc1-cdbb-44e5-b902-a37834858a0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:21.028258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:21.029129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:21.029965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:21.029965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:21.029965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:21.029965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 265/531] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:21.030190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074fc1-cdbb-44e5-b902-a37834858a0e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:21.039474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ae1c6-2097-483c-a425-24ed92ad014a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:21.041283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ae1c6-2097-483c-a425-24ed92ad014a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9c6ae1c6-2097-483c-a425-24ed92ad014a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:21.057811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:21.058769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:21.059590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:21.059590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:21.059590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:21.061984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 267/532] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:21.063380 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ae1c6-2097-483c-a425-24ed92ad014a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:21.081095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:21.185493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074fc1-cdbb-44e5-b902-a37834858a0e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:21.186233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ae1c6-2097-483c-a425-24ed92ad014a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:21.224331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd26ac-534f-4df7-b7e3-352d48d39dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:21.225334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd26ac-534f-4df7-b7e3-352d48d39dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:21.225822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd26ac-534f-4df7-b7e3-352d48d39dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:21.226286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd26ac-534f-4df7-b7e3-352d48d39dc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:21.227264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd26ac-534f-4df7-b7e3-352d48d39dc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:21.227877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd26ac-534f-4df7-b7e3-352d48d39dc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:21.230174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd26ac-534f-4df7-b7e3-352d48d39dc0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:21.230462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fae0b4-ca94-4e06-9609-7cc830948ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:21.231059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae0b4-ca94-4e06-9609-7cc830948ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:21.231333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae0b4-ca94-4e06-9609-7cc830948ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:21.231597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fae0b4-ca94-4e06-9609-7cc830948ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:21.232337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fae0b4-ca94-4e06-9609-7cc830948ce6 None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=5c84b440821446e3a7ce97b2c68f815e) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:21.232636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fae0b4-ca94-4e06-9609-7cc830948ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:21.234942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fae0b4-ca94-4e06-9609-7cc830948ce6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:21.247990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 266/533] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:21 2022] GET /identity/v3/role_assignments?user.id=5c84b440821446e3a7ce97b2c68f815e&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:21.249353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fae0b4-ca94-4e06-9609-7cc830948ce6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:21.255614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 268/534] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:21.260823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd26ac-534f-4df7-b7e3-352d48d39dc0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:21.317789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:21.384099 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:21.469997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd26ac-534f-4df7-b7e3-352d48d39dc0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:21.484843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fae0b4-ca94-4e06-9609-7cc830948ce6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:21.517490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5398a-cd2b-41d6-8f8d-9f136c913c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:21.518129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5398a-cd2b-41d6-8f8d-9f136c913c06 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:21.518411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5398a-cd2b-41d6-8f8d-9f136c913c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:21.518685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5398a-cd2b-41d6-8f8d-9f136c913c06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:21.522962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92151233-64bd-4580-ba64-f02e4bd6d793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:21.523570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92151233-64bd-4580-ba64-f02e4bd6d793 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:21.523868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92151233-64bd-4580-ba64-f02e4bd6d793 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:21.524136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92151233-64bd-4580-ba64-f02e4bd6d793 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:21.524801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92151233-64bd-4580-ba64-f02e4bd6d793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:21.525212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92151233-64bd-4580-ba64-f02e4bd6d793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:21.527864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92151233-64bd-4580-ba64-f02e4bd6d793 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:21.541661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 267/535] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:21.546963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92151233-64bd-4580-ba64-f02e4bd6d793 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:21.548924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5398a-cd2b-41d6-8f8d-9f136c913c06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:21.550485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5398a-cd2b-41d6-8f8d-9f136c913c06 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-33c5398a-cd2b-41d6-8f8d-9f136c913c06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:21.582578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:21.583534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:21.584425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:21.584425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:21.584425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:21.584425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:21.584425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:21.584425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:21.584425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:21.584425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:21.584425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 269/536] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:21.593958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5398a-cd2b-41d6-8f8d-9f136c913c06 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:21.649367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:21.660004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92151233-64bd-4580-ba64-f02e4bd6d793 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:21.696466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f39d2f-ebbd-4294-af44-9f7f14478864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:21.703884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f39d2f-ebbd-4294-af44-9f7f14478864 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:21.703884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f39d2f-ebbd-4294-af44-9f7f14478864 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:21.703884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f39d2f-ebbd-4294-af44-9f7f14478864 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:21.716492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f39d2f-ebbd-4294-af44-9f7f14478864 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:21.722250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f39d2f-ebbd-4294-af44-9f7f14478864 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-55f39d2f-ebbd-4294-af44-9f7f14478864 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:21.738181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:21.739222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:21.740174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:21.740174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:21.740174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:21.740174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:21.740174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:21.740174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:21.740174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:21.740174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:21.742986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 268/537] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:21.755702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f39d2f-ebbd-4294-af44-9f7f14478864 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:21.776304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5398a-cd2b-41d6-8f8d-9f136c913c06 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:21.807466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:21.818437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17e9d4-6e50-46dc-839e-e766fc67fee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:21.819126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17e9d4-6e50-46dc-839e-e766fc67fee4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:21.819472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17e9d4-6e50-46dc-839e-e766fc67fee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:21.819916 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17e9d4-6e50-46dc-839e-e766fc67fee4 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:21.821077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17e9d4-6e50-46dc-839e-e766fc67fee4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:21.821455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17e9d4-6e50-46dc-839e-e766fc67fee4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:21.823571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17e9d4-6e50-46dc-839e-e766fc67fee4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:21.837840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 270/538] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:21.844618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17e9d4-6e50-46dc-839e-e766fc67fee4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:21.961890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17e9d4-6e50-46dc-839e-e766fc67fee4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:22.001724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1fbca-1628-41a7-b57c-89bfb8450622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:22.002250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1fbca-1628-41a7-b57c-89bfb8450622 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:22.002529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1fbca-1628-41a7-b57c-89bfb8450622 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:22.002799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1fbca-1628-41a7-b57c-89bfb8450622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:22.003597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1fbca-1628-41a7-b57c-89bfb8450622 None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=cd66d5d9989449729cec7e60cdaf0949) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:22.003931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1fbca-1628-41a7-b57c-89bfb8450622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:22.005960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1fbca-1628-41a7-b57c-89bfb8450622 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:22.013600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 271/539] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:21 2022] GET /identity/v3/role_assignments?user.id=cd66d5d9989449729cec7e60cdaf0949&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:22.031956 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f39d2f-ebbd-4294-af44-9f7f14478864 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:22.076162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d16eaf-cb61-48cd-8ba4-da78e0104208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:22.076813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d16eaf-cb61-48cd-8ba4-da78e0104208 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:22.077093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d16eaf-cb61-48cd-8ba4-da78e0104208 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:22.077370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d16eaf-cb61-48cd-8ba4-da78e0104208 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:22.078092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d16eaf-cb61-48cd-8ba4-da78e0104208 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:22.078399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d16eaf-cb61-48cd-8ba4-da78e0104208 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:22.079901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d16eaf-cb61-48cd-8ba4-da78e0104208 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:22.110450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 269/540] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:22.116882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1fbca-1628-41a7-b57c-89bfb8450622 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:22.188448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:22.244248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1fbca-1628-41a7-b57c-89bfb8450622 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:22.290549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d10a8d-07f2-452b-8b7f-1aed6f0811f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:22.291589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d10a8d-07f2-452b-8b7f-1aed6f0811f6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:22.292093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d10a8d-07f2-452b-8b7f-1aed6f0811f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:22.292604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d10a8d-07f2-452b-8b7f-1aed6f0811f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:22.293554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d10a8d-07f2-452b-8b7f-1aed6f0811f6 None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=56a94ccca15c462c9e7a5e88b907f420) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:22.294066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d10a8d-07f2-452b-8b7f-1aed6f0811f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:22.295691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d10a8d-07f2-452b-8b7f-1aed6f0811f6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:22.306948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 272/541] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:22 2022] GET /identity/v3/role_assignments?user.id=56a94ccca15c462c9e7a5e88b907f420&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:22.684223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0e13d-0687-4294-864f-c8f7dbca564b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:22.684607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0e13d-0687-4294-864f-c8f7dbca564b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:22.684782 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0e13d-0687-4294-864f-c8f7dbca564b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:22.686416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 270/542] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:22.688813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f59d5e-8769-425f-b9ca-c7d7af56342b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:22.689347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f59d5e-8769-425f-b9ca-c7d7af56342b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:22.689830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f59d5e-8769-425f-b9ca-c7d7af56342b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:22.692317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29537faf-4ab5-452b-84b2-70a89d04f4e7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:22.692831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29537faf-4ab5-452b-84b2-70a89d04f4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:22.693291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29537faf-4ab5-452b-84b2-70a89d04f4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:22.697091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 273/543] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:22.703183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e33e-39da-4322-81ff-ecd52b5f6673 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:22.703470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e33e-39da-4322-81ff-ecd52b5f6673 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:22.703746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87e33e-39da-4322-81ff-ecd52b5f6673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:22.724358 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-29537faf-4ab5-452b-84b2-70a89d04f4e7 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:22.724948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5a87e33e-39da-4322-81ff-ecd52b5f6673 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:22.726145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5a87e33e-39da-4322-81ff-ecd52b5f6673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:22.726354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-29537faf-4ab5-452b-84b2-70a89d04f4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:22.735737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:22.815981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 274/544] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:07:22.879386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 271/545] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:23.226994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87e33e-39da-4322-81ff-ecd52b5f6673 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:23.280165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:23.288446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29537faf-4ab5-452b-84b2-70a89d04f4e7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:23.342274 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87e33e-39da-4322-81ff-ecd52b5f6673 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:23.382340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a734966-814c-482f-b3c9-01cdcc3771c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:23.382982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a734966-814c-482f-b3c9-01cdcc3771c1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:23.383267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a734966-814c-482f-b3c9-01cdcc3771c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:23.383545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a734966-814c-482f-b3c9-01cdcc3771c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:23.395462 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a734966-814c-482f-b3c9-01cdcc3771c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0a734966-814c-482f-b3c9-01cdcc3771c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:23.401648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:23.402560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:23.403354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 275/546] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:23.410699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a734966-814c-482f-b3c9-01cdcc3771c1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:23.516637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29537faf-4ab5-452b-84b2-70a89d04f4e7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:23.540671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a734966-814c-482f-b3c9-01cdcc3771c1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:23.572545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35e0b-1424-4911-b474-1209f5cab450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:23.573197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35e0b-1424-4911-b474-1209f5cab450 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:23.573481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35e0b-1424-4911-b474-1209f5cab450 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:23.573747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35e0b-1424-4911-b474-1209f5cab450 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:23.573853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:23.585606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82fd92-a16f-4d2d-b2c4-e00686f7fe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:23.586651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82fd92-a16f-4d2d-b2c4-e00686f7fe88 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:23.587114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82fd92-a16f-4d2d-b2c4-e00686f7fe88 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:23.587566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82fd92-a16f-4d2d-b2c4-e00686f7fe88 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:23.589084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82fd92-a16f-4d2d-b2c4-e00686f7fe88 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:23.589606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82fd92-a16f-4d2d-b2c4-e00686f7fe88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:23.591493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f82fd92-a16f-4d2d-b2c4-e00686f7fe88 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:23.601886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 276/547] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:23.603174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd35e0b-1424-4911-b474-1209f5cab450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3dd35e0b-1424-4911-b474-1209f5cab450 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:23.608774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:23.609672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:23.610595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:23.614248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 272/548] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:23.615409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82fd92-a16f-4d2d-b2c4-e00686f7fe88 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:23.619673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35e0b-1424-4911-b474-1209f5cab450 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:23.627529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:23.735482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82fd92-a16f-4d2d-b2c4-e00686f7fe88 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:23.780000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2ac1b-7694-42a6-b63a-175621e96f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:23.780565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2ac1b-7694-42a6-b63a-175621e96f1c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:23.780847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2ac1b-7694-42a6-b63a-175621e96f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:23.781122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2ac1b-7694-42a6-b63a-175621e96f1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:23.803015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2ac1b-7694-42a6-b63a-175621e96f1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:23.804506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2ac1b-7694-42a6-b63a-175621e96f1c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-efe2ac1b-7694-42a6-b63a-175621e96f1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:23.825657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:23.827989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:23.829008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:23.829008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:23.829008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:23.829008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 277/549] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:23.835994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2ac1b-7694-42a6-b63a-175621e96f1c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:23.872856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:23.873040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35e0b-1424-4911-b474-1209f5cab450 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:23.926725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d24dc-e13c-4f2c-bf11-151a7ca7ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:23.927395 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d24dc-e13c-4f2c-bf11-151a7ca7ef11 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:23.927682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d24dc-e13c-4f2c-bf11-151a7ca7ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:23.928048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d24dc-e13c-4f2c-bf11-151a7ca7ef11 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:23.928804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d24dc-e13c-4f2c-bf11-151a7ca7ef11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:23.929110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d24dc-e13c-4f2c-bf11-151a7ca7ef11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:23.930760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d24dc-e13c-4f2c-bf11-151a7ca7ef11 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:23.947824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 273/550] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:23.947824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a361c-ad55-4ae8-8aa9-fb1dc6b282a5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:23.947824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a361c-ad55-4ae8-8aa9-fb1dc6b282a5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:23.947824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a361c-ad55-4ae8-8aa9-fb1dc6b282a5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:23.947824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 274/551] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:23.947824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a361c-ad55-4ae8-8aa9-fb1dc6b282a5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:23.981354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2ac1b-7694-42a6-b63a-175621e96f1c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:24.017877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb7c7b-1e7c-4e78-9783-ecebe623826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:24.018866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb7c7b-1e7c-4e78-9783-ecebe623826e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.019395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb7c7b-1e7c-4e78-9783-ecebe623826e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.019901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb7c7b-1e7c-4e78-9783-ecebe623826e None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.020813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb7c7b-1e7c-4e78-9783-ecebe623826e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:24.021334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb7c7b-1e7c-4e78-9783-ecebe623826e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:24.022962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb7c7b-1e7c-4e78-9783-ecebe623826e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:24.028805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:24.038379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 278/552] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:24.040091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df015a7-948c-46d7-8cd2-d666fd927d90 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.040585 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df015a7-948c-46d7-8cd2-d666fd927d90 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.041095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df015a7-948c-46d7-8cd2-d666fd927d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.060188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9df015a7-948c-46d7-8cd2-d666fd927d90 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:24.061748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9df015a7-948c-46d7-8cd2-d666fd927d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:24.076591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a361c-ad55-4ae8-8aa9-fb1dc6b282a5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:24.123097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f8ebb-698f-42bd-a1e7-e2df060ba3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:24.124051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f8ebb-698f-42bd-a1e7-e2df060ba3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.124514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f8ebb-698f-42bd-a1e7-e2df060ba3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.124934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f8ebb-698f-42bd-a1e7-e2df060ba3f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.138915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f8ebb-698f-42bd-a1e7-e2df060ba3f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:24.140795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f8ebb-698f-42bd-a1e7-e2df060ba3f0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:24.147778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 279/553] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:24.147778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df015a7-948c-46d7-8cd2-d666fd927d90 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:24.159177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-be5f8ebb-698f-42bd-a1e7-e2df060ba3f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:24.168753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:24.169789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:24.170647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:24.170647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:24.170647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:24.177421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 275/554] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:24.178538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f8ebb-698f-42bd-a1e7-e2df060ba3f0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:24.277329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df015a7-948c-46d7-8cd2-d666fd927d90 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:24.318570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e393b8d-7d17-4cde-8ef7-7ccf08feb766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:24.319262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393b8d-7d17-4cde-8ef7-7ccf08feb766 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.319549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393b8d-7d17-4cde-8ef7-7ccf08feb766 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.319850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393b8d-7d17-4cde-8ef7-7ccf08feb766 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.331082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e393b8d-7d17-4cde-8ef7-7ccf08feb766 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:24.331082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e393b8d-7d17-4cde-8ef7-7ccf08feb766 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2e393b8d-7d17-4cde-8ef7-7ccf08feb766 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:24.349517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:24.350698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:24.351919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:24.354981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 280/555] 10.222.0.71 () {56 vars in 1093 bytes} [Wed Sep 21 12:07:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:24.360651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e393b8d-7d17-4cde-8ef7-7ccf08feb766 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:24.446536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f8ebb-698f-42bd-a1e7-e2df060ba3f0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:24.447057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:24.478267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:24.491315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e393b8d-7d17-4cde-8ef7-7ccf08feb766 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:24.499461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d4ae0-cdda-4553-9973-ef60db9d735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:24.500293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d4ae0-cdda-4553-9973-ef60db9d735c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.500714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d4ae0-cdda-4553-9973-ef60db9d735c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.501318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d4ae0-cdda-4553-9973-ef60db9d735c None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.502382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d4ae0-cdda-4553-9973-ef60db9d735c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:24.503033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d4ae0-cdda-4553-9973-ef60db9d735c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:24.504777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d4ae0-cdda-4553-9973-ef60db9d735c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:24.518468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 276/556] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:24.524230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8c35-9e2f-4a60-a67c-a938fc041ff2 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.524810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8c35-9e2f-4a60-a67c-a938fc041ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.525317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8c35-9e2f-4a60-a67c-a938fc041ff2 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.527685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 277/557] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:24.529274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8c35-9e2f-4a60-a67c-a938fc041ff2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:24.533189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdda125d-8a1e-4a4c-b292-d073ffb1a9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:24.533810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda125d-8a1e-4a4c-b292-d073ffb1a9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.534095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda125d-8a1e-4a4c-b292-d073ffb1a9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.534371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdda125d-8a1e-4a4c-b292-d073ffb1a9a4 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.536137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda125d-8a1e-4a4c-b292-d073ffb1a9a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:24.536456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda125d-8a1e-4a4c-b292-d073ffb1a9a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:24.538020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdda125d-8a1e-4a4c-b292-d073ffb1a9a4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:24.561284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 281/558] 10.222.0.71 () {56 vars in 1133 bytes} [Wed Sep 21 12:07:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:24.562765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4e424-22ad-4235-b2e9-b8eec1d26d72 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.563055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4e424-22ad-4235-b2e9-b8eec1d26d72 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.563322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4e424-22ad-4235-b2e9-b8eec1d26d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.582289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4dc4e424-22ad-4235-b2e9-b8eec1d26d72 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:24.583411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4dc4e424-22ad-4235-b2e9-b8eec1d26d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:24.652419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8c35-9e2f-4a60-a67c-a938fc041ff2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:24.652729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 282/559] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:24.653736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4e424-22ad-4235-b2e9-b8eec1d26d72 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:24.699003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831349cd-663f-4610-9447-b0ad3fbbe209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:24.699724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831349cd-663f-4610-9447-b0ad3fbbe209 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.700097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831349cd-663f-4610-9447-b0ad3fbbe209 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.700436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831349cd-663f-4610-9447-b0ad3fbbe209 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.709644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831349cd-663f-4610-9447-b0ad3fbbe209 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:24.711055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831349cd-663f-4610-9447-b0ad3fbbe209 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:24.723491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-831349cd-663f-4610-9447-b0ad3fbbe209 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:24.727146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:24.728158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:24.729690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:24.733980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 278/560] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:24.734896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831349cd-663f-4610-9447-b0ad3fbbe209 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:24.773790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4e424-22ad-4235-b2e9-b8eec1d26d72 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:24.803143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:24.813043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d79f96-537c-46d7-b55a-24a6487d7529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:24.814263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d79f96-537c-46d7-b55a-24a6487d7529 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.814714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d79f96-537c-46d7-b55a-24a6487d7529 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.815155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d79f96-537c-46d7-b55a-24a6487d7529 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.826739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d79f96-537c-46d7-b55a-24a6487d7529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-35d79f96-537c-46d7-b55a-24a6487d7529 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:24.832251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:24.833222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:24.834049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:24.836596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 283/561] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:24.837699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d79f96-537c-46d7-b55a-24a6487d7529 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:24.943909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831349cd-663f-4610-9447-b0ad3fbbe209 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:24.949587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d79f96-537c-46d7-b55a-24a6487d7529 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:24.968896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:24.987878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce776e-14d5-41b1-a384-024961ad693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:24.988851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce776e-14d5-41b1-a384-024961ad693e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:24.989429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce776e-14d5-41b1-a384-024961ad693e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:24.989919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce776e-14d5-41b1-a384-024961ad693e None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:24.990943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce776e-14d5-41b1-a384-024961ad693e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:24.991439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce776e-14d5-41b1-a384-024961ad693e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:24.993231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fce776e-14d5-41b1-a384-024961ad693e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.004181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9c95b-3836-4148-a2d2-39e4a1364b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.004818 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9c95b-3836-4148-a2d2-39e4a1364b93 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.005093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9c95b-3836-4148-a2d2-39e4a1364b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.005361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9c95b-3836-4148-a2d2-39e4a1364b93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.012866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 284/562] 10.222.0.71 () {56 vars in 1135 bytes} [Wed Sep 21 12:07:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:25.013947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce776e-14d5-41b1-a384-024961ad693e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.021173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9c95b-3836-4148-a2d2-39e4a1364b93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.022645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9c95b-3836-4148-a2d2-39e4a1364b93 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-76d9c95b-3836-4148-a2d2-39e4a1364b93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:25.039406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:25.040660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:25.042106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:25.042106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:25.042106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:25.042106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 279/563] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:25.042106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9c95b-3836-4148-a2d2-39e4a1364b93 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.120841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fce776e-14d5-41b1-a384-024961ad693e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.129433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:25.141767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9c95b-3836-4148-a2d2-39e4a1364b93 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.154783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cba3a-b918-4262-9da9-f02d81cd1114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.155398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cba3a-b918-4262-9da9-f02d81cd1114 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.155678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cba3a-b918-4262-9da9-f02d81cd1114 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.155986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cba3a-b918-4262-9da9-f02d81cd1114 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.156864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cba3a-b918-4262-9da9-f02d81cd1114 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:25.157167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cba3a-b918-4262-9da9-f02d81cd1114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.158964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cba3a-b918-4262-9da9-f02d81cd1114 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.168163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 285/564] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:25.168996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cba3a-b918-4262-9da9-f02d81cd1114 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.183968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2d769-c527-4cef-b95b-82e743fa59e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.184779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2d769-c527-4cef-b95b-82e743fa59e4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.185063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2d769-c527-4cef-b95b-82e743fa59e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.185336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2d769-c527-4cef-b95b-82e743fa59e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.195312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2d769-c527-4cef-b95b-82e743fa59e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-16e2d769-c527-4cef-b95b-82e743fa59e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:25.201190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:25.202138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:25.202955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 280/565] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:25.203647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b985be-83ab-42a4-99a5-f2807a054a04 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.203722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b985be-83ab-42a4-99a5-f2807a054a04 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.204145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b985be-83ab-42a4-99a5-f2807a054a04 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.207897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 281/566] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:25.207897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b985be-83ab-42a4-99a5-f2807a054a04 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.210011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:25.271604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cba3a-b918-4262-9da9-f02d81cd1114 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.306269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669610b8-a6a0-4956-9eff-864475a673e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.307790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669610b8-a6a0-4956-9eff-864475a673e8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.308071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669610b8-a6a0-4956-9eff-864475a673e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.308169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669610b8-a6a0-4956-9eff-864475a673e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.309445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b985be-83ab-42a4-99a5-f2807a054a04 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.342516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6724368-0a3b-4bfe-b97f-f9bcce6c3e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.343108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6724368-0a3b-4bfe-b97f-f9bcce6c3e6d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.343388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6724368-0a3b-4bfe-b97f-f9bcce6c3e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.343654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6724368-0a3b-4bfe-b97f-f9bcce6c3e6d None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.344315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6724368-0a3b-4bfe-b97f-f9bcce6c3e6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:25.344611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6724368-0a3b-4bfe-b97f-f9bcce6c3e6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.345970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6724368-0a3b-4bfe-b97f-f9bcce6c3e6d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.358467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 282/567] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:25.359288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6724368-0a3b-4bfe-b97f-f9bcce6c3e6d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.371176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:25.376955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669610b8-a6a0-4956-9eff-864475a673e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.378404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669610b8-a6a0-4956-9eff-864475a673e8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.414593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:25.459550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6724368-0a3b-4bfe-b97f-f9bcce6c3e6d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.476698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 286/568] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:25.477542 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669610b8-a6a0-4956-9eff-864475a673e8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.494861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417284e2-d9af-4876-8e2b-d39b3321305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.495470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417284e2-d9af-4876-8e2b-d39b3321305f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.495781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417284e2-d9af-4876-8e2b-d39b3321305f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.496054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417284e2-d9af-4876-8e2b-d39b3321305f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.507231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417284e2-d9af-4876-8e2b-d39b3321305f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.508708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417284e2-d9af-4876-8e2b-d39b3321305f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-417284e2-d9af-4876-8e2b-d39b3321305f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:25.523950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:25.524941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:25.526441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:25.526441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:25.526441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:25.526441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 283/569] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:25.526794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885799f6-e39a-4ba6-a3c1-1b2a1e9ac176 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.527038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885799f6-e39a-4ba6-a3c1-1b2a1e9ac176 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.527301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885799f6-e39a-4ba6-a3c1-1b2a1e9ac176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.545967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-885799f6-e39a-4ba6-a3c1-1b2a1e9ac176 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:25.547154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-885799f6-e39a-4ba6-a3c1-1b2a1e9ac176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:25.581075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669610b8-a6a0-4956-9eff-864475a673e8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.604845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:25.615594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6314997-5d0b-452c-8026-ba85e82253a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.625946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 284/570] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:25.626108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6314997-5d0b-452c-8026-ba85e82253a1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.626108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6314997-5d0b-452c-8026-ba85e82253a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.626108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6314997-5d0b-452c-8026-ba85e82253a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.626405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885799f6-e39a-4ba6-a3c1-1b2a1e9ac176 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.626510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6314997-5d0b-452c-8026-ba85e82253a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:25.626510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6314997-5d0b-452c-8026-ba85e82253a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.626510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6314997-5d0b-452c-8026-ba85e82253a1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.626510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:25.629956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 287/571] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:25.629956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6314997-5d0b-452c-8026-ba85e82253a1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.725243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885799f6-e39a-4ba6-a3c1-1b2a1e9ac176 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.731926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6314997-5d0b-452c-8026-ba85e82253a1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.766821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2cba6-278e-4eee-8332-6e7eb22b8d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.767459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2cba6-278e-4eee-8332-6e7eb22b8d5f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.767740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2cba6-278e-4eee-8332-6e7eb22b8d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.768033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2cba6-278e-4eee-8332-6e7eb22b8d5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.773939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679e899-bb92-4a5e-9d8f-57469046b702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.774848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2cba6-278e-4eee-8332-6e7eb22b8d5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.775109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679e899-bb92-4a5e-9d8f-57469046b702 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.775601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679e899-bb92-4a5e-9d8f-57469046b702 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.776081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3679e899-bb92-4a5e-9d8f-57469046b702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.776392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2cba6-278e-4eee-8332-6e7eb22b8d5f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.787337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679e899-bb92-4a5e-9d8f-57469046b702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f8b2cba6-278e-4eee-8332-6e7eb22b8d5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:25.787509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:25.788367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:25.794412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:25.794412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:25.794412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:25.794412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:25.794412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:25.794412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:25.794412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:25.794412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:25.794834 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3679e899-bb92-4a5e-9d8f-57469046b702 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.794916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 285/572] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:25.794916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2cba6-278e-4eee-8332-6e7eb22b8d5f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3679e899-bb92-4a5e-9d8f-57469046b702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:25.808574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:25.809530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:25.810459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:25.810459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:25.810459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:25.810459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 288/573] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:25.810459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679e899-bb92-4a5e-9d8f-57469046b702 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.854519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:25.870836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:25.909394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2cba6-278e-4eee-8332-6e7eb22b8d5f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.924300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3679e899-bb92-4a5e-9d8f-57469046b702 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:25.943011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79d5ea-bb10-4128-b814-f079364c5e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.943695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79d5ea-bb10-4128-b814-f079364c5e5c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.944069 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79d5ea-bb10-4128-b814-f079364c5e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.944343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc79d5ea-bb10-4128-b814-f079364c5e5c None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.944979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79d5ea-bb10-4128-b814-f079364c5e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:25.945281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79d5ea-bb10-4128-b814-f079364c5e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.946642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc79d5ea-bb10-4128-b814-f079364c5e5c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.957489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 286/574] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:25.958372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79d5ea-bb10-4128-b814-f079364c5e5c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:25.958994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c209e-977a-4138-990b-3fc53d70fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:25.959825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c209e-977a-4138-990b-3fc53d70fb61 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:25.960112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c209e-977a-4138-990b-3fc53d70fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:25.960391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c209e-977a-4138-990b-3fc53d70fb61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:25.971571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c209e-977a-4138-990b-3fc53d70fb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:25.973235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c209e-977a-4138-990b-3fc53d70fb61 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f75c209e-977a-4138-990b-3fc53d70fb61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:25.987510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:25.988480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:25.989403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:25.989403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:25.989403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:25.989403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 289/575] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:25.990591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c209e-977a-4138-990b-3fc53d70fb61 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.060837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc79d5ea-bb10-4128-b814-f079364c5e5c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.071017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:26.091811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c209e-977a-4138-990b-3fc53d70fb61 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.094173 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:26.095006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab6179-5d73-4eca-b578-489abf78f13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.095619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab6179-5d73-4eca-b578-489abf78f13d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.095926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab6179-5d73-4eca-b578-489abf78f13d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.096200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab6179-5d73-4eca-b578-489abf78f13d None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.096892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab6179-5d73-4eca-b578-489abf78f13d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:26.097199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab6179-5d73-4eca-b578-489abf78f13d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.098647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bab6179-5d73-4eca-b578-489abf78f13d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.111083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 287/576] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:26.111717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab6179-5d73-4eca-b578-489abf78f13d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.130610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f5593-4404-4f72-8042-b75b922cd199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.131221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f5593-4404-4f72-8042-b75b922cd199 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.131507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f5593-4404-4f72-8042-b75b922cd199 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.131798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f5593-4404-4f72-8042-b75b922cd199 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.132444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f5593-4404-4f72-8042-b75b922cd199 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:26.132750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f5593-4404-4f72-8042-b75b922cd199 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.134124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f5593-4404-4f72-8042-b75b922cd199 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.146066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 290/577] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:26.146720 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f5593-4404-4f72-8042-b75b922cd199 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.214616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab6179-5d73-4eca-b578-489abf78f13d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.246115 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f5593-4404-4f72-8042-b75b922cd199 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.248444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152e335-5902-44ef-88e0-c15d70a25e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.249063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e335-5902-44ef-88e0-c15d70a25e5a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.249342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e335-5902-44ef-88e0-c15d70a25e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.249609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152e335-5902-44ef-88e0-c15d70a25e5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.260298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152e335-5902-44ef-88e0-c15d70a25e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a152e335-5902-44ef-88e0-c15d70a25e5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:26.265547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:26.266652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 288/578] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:26.272681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152e335-5902-44ef-88e0-c15d70a25e5a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.284264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303f31e-99bb-4515-832d-d38081272e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.284919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303f31e-99bb-4515-832d-d38081272e93 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.285198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303f31e-99bb-4515-832d-d38081272e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.285474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303f31e-99bb-4515-832d-d38081272e93 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.292837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303f31e-99bb-4515-832d-d38081272e93 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.294289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303f31e-99bb-4515-832d-d38081272e93 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.299262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4303f31e-99bb-4515-832d-d38081272e93 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:26.306232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:26.308702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 291/579] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:26.310348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303f31e-99bb-4515-832d-d38081272e93 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.337386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:26.374893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152e335-5902-44ef-88e0-c15d70a25e5a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.408072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562d573-a006-462d-b9dc-521b2e158dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.408669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562d573-a006-462d-b9dc-521b2e158dac None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.408950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562d573-a006-462d-b9dc-521b2e158dac None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.409226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0562d573-a006-462d-b9dc-521b2e158dac None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.409859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562d573-a006-462d-b9dc-521b2e158dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:26.410169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562d573-a006-462d-b9dc-521b2e158dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.411564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0562d573-a006-462d-b9dc-521b2e158dac None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.416552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303f31e-99bb-4515-832d-d38081272e93 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.423390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 289/580] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:26.424239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562d573-a006-462d-b9dc-521b2e158dac None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.452198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7304947-850f-421e-aca9-7c70a7f49df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.452862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7304947-850f-421e-aca9-7c70a7f49df6 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.453364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7304947-850f-421e-aca9-7c70a7f49df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.453477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7304947-850f-421e-aca9-7c70a7f49df6 None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/463487449c1d4aada1b5f279c959b0e6/roles/fcc43e29f89e4c6cb1690a36d91ec4c1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.468731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7304947-850f-421e-aca9-7c70a7f49df6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=463487449c1d4aada1b5f279c959b0e6, role_id=fcc43e29f89e4c6cb1690a36d91ec4c1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.470114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7304947-850f-421e-aca9-7c70a7f49df6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.482877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 292/581] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:07:26 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/463487449c1d4aada1b5f279c959b0e6/roles/fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:26.483688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7304947-850f-421e-aca9-7c70a7f49df6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.530722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:26.542809 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0562d573-a006-462d-b9dc-521b2e158dac None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.583081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:26.583669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea88f2-8cb4-4124-996d-9b749a213593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.584355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea88f2-8cb4-4124-996d-9b749a213593 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.584626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea88f2-8cb4-4124-996d-9b749a213593 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.584897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea88f2-8cb4-4124-996d-9b749a213593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.593156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea88f2-8cb4-4124-996d-9b749a213593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.594648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea88f2-8cb4-4124-996d-9b749a213593 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d5ea88f2-8cb4-4124-996d-9b749a213593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:26.607203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:26.608232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:26.609108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:26.609108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:26.609108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:26.609108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:26.609108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:26.609108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:26.609108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:26.609108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:26.609108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 290/582] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:26.609602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea88f2-8cb4-4124-996d-9b749a213593 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.620805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7304947-850f-421e-aca9-7c70a7f49df6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.666599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1efa-1a52-4dee-ae76-ee25c0cadbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.667464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1efa-1a52-4dee-ae76-ee25c0cadbfe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.668145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1efa-1a52-4dee-ae76-ee25c0cadbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.668799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1efa-1a52-4dee-ae76-ee25c0cadbfe None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.669270 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a1efa-1a52-4dee-ae76-ee25c0cadbfe None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:26.669574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a1efa-1a52-4dee-ae76-ee25c0cadbfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.670968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a1efa-1a52-4dee-ae76-ee25c0cadbfe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.680456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 293/583] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:26.681298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1efa-1a52-4dee-ae76-ee25c0cadbfe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.728924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea88f2-8cb4-4124-996d-9b749a213593 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.772783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ed89f-5f9a-4732-a9d8-4ecf6536a6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.773397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ed89f-5f9a-4732-a9d8-4ecf6536a6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.773676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ed89f-5f9a-4732-a9d8-4ecf6536a6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.773945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ed89f-5f9a-4732-a9d8-4ecf6536a6e4 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.774677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ed89f-5f9a-4732-a9d8-4ecf6536a6e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:26.774981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ed89f-5f9a-4732-a9d8-4ecf6536a6e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.776997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ed89f-5f9a-4732-a9d8-4ecf6536a6e4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.793440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1efa-1a52-4dee-ae76-ee25c0cadbfe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.797126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 291/584] 10.222.0.71 () {56 vars in 1137 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:26.797980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ed89f-5f9a-4732-a9d8-4ecf6536a6e4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.813620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:26.830571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5ecc7-5294-44ab-aad5-da6d6b2ccb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.831208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5ecc7-5294-44ab-aad5-da6d6b2ccb7f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.831497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5ecc7-5294-44ab-aad5-da6d6b2ccb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.831783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5ecc7-5294-44ab-aad5-da6d6b2ccb7f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.835557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:26.842491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5ecc7-5294-44ab-aad5-da6d6b2ccb7f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.844638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5ecc7-5294-44ab-aad5-da6d6b2ccb7f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3ef5ecc7-5294-44ab-aad5-da6d6b2ccb7f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:26.858359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:26.859322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 294/585] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:26.860278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5ecc7-5294-44ab-aad5-da6d6b2ccb7f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.909998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ed89f-5f9a-4732-a9d8-4ecf6536a6e4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:26.945433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f02bba-6511-42cf-92a2-8bb691770cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:26.946046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f02bba-6511-42cf-92a2-8bb691770cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:26.946326 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f02bba-6511-42cf-92a2-8bb691770cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:26.946599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f02bba-6511-42cf-92a2-8bb691770cc7 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:26.947839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f02bba-6511-42cf-92a2-8bb691770cc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:26.948170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f02bba-6511-42cf-92a2-8bb691770cc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:26.949620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f02bba-6511-42cf-92a2-8bb691770cc7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:26.968019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 292/586] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:26.968740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f02bba-6511-42cf-92a2-8bb691770cc7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:26.975827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5ecc7-5294-44ab-aad5-da6d6b2ccb7f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.010439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080d961-155a-47c0-b187-6db0cfe47e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.010987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080d961-155a-47c0-b187-6db0cfe47e72 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.011265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080d961-155a-47c0-b187-6db0cfe47e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.011535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080d961-155a-47c0-b187-6db0cfe47e72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.022442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080d961-155a-47c0-b187-6db0cfe47e72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.023945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c080d961-155a-47c0-b187-6db0cfe47e72 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c080d961-155a-47c0-b187-6db0cfe47e72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:27.039299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:27.040495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:27.041599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:27.041599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:27.041599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:27.045226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 295/587] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:27.045656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:27.045941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080d961-155a-47c0-b187-6db0cfe47e72 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.086887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f02bba-6511-42cf-92a2-8bb691770cc7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.103864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:27.123573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82d40a-b9d9-4cfc-a1ca-47dfda6010bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.124244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82d40a-b9d9-4cfc-a1ca-47dfda6010bc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.124527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82d40a-b9d9-4cfc-a1ca-47dfda6010bc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.124795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82d40a-b9d9-4cfc-a1ca-47dfda6010bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.136999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82d40a-b9d9-4cfc-a1ca-47dfda6010bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.138808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82d40a-b9d9-4cfc-a1ca-47dfda6010bc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ea82d40a-b9d9-4cfc-a1ca-47dfda6010bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:27.153362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:27.154337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:27.155238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:27.155238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:27.155238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:27.155238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 293/588] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:27.155238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82d40a-b9d9-4cfc-a1ca-47dfda6010bc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.167728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080d961-155a-47c0-b187-6db0cfe47e72 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.212901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045fe4-2bdd-4d87-b56c-74b21b57118c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.213507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045fe4-2bdd-4d87-b56c-74b21b57118c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.213791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045fe4-2bdd-4d87-b56c-74b21b57118c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.214068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045fe4-2bdd-4d87-b56c-74b21b57118c None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.214800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045fe4-2bdd-4d87-b56c-74b21b57118c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:27.215109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045fe4-2bdd-4d87-b56c-74b21b57118c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.216840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045fe4-2bdd-4d87-b56c-74b21b57118c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.238619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 296/589] 10.222.0.71 () {56 vars in 1139 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:27.238619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045fe4-2bdd-4d87-b56c-74b21b57118c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.281853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82d40a-b9d9-4cfc-a1ca-47dfda6010bc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.306947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:27.321452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ed947-f965-4d45-9cfe-2398700f1985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.322734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ed947-f965-4d45-9cfe-2398700f1985 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.323113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ed947-f965-4d45-9cfe-2398700f1985 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.323476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ed947-f965-4d45-9cfe-2398700f1985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.324334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ed947-f965-4d45-9cfe-2398700f1985 None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=e0bf75c730c946bc9168dd418e7af637) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:27.324733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ed947-f965-4d45-9cfe-2398700f1985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.326255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ed947-f965-4d45-9cfe-2398700f1985 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.334363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 294/590] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/role_assignments?user.id=e0bf75c730c946bc9168dd418e7af637&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:27.335322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ed947-f965-4d45-9cfe-2398700f1985 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.359890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045fe4-2bdd-4d87-b56c-74b21b57118c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.374589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:27.419295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee490c4e-1711-4b27-831e-49801e467aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.420234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee490c4e-1711-4b27-831e-49801e467aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.420664 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee490c4e-1711-4b27-831e-49801e467aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.421087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee490c4e-1711-4b27-831e-49801e467aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.422237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee490c4e-1711-4b27-831e-49801e467aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:27.422719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee490c4e-1711-4b27-831e-49801e467aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.425318 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee490c4e-1711-4b27-831e-49801e467aa7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.446324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 297/591] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:27.448145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee490c4e-1711-4b27-831e-49801e467aa7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.472085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ed947-f965-4d45-9cfe-2398700f1985 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.516388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78410220-273a-44a9-aafc-3cadcc0d5950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.517010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78410220-273a-44a9-aafc-3cadcc0d5950 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.517292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78410220-273a-44a9-aafc-3cadcc0d5950 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.517564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78410220-273a-44a9-aafc-3cadcc0d5950 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.518217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78410220-273a-44a9-aafc-3cadcc0d5950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:27.518685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78410220-273a-44a9-aafc-3cadcc0d5950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.520688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78410220-273a-44a9-aafc-3cadcc0d5950 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.538822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 295/592] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:27.539626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78410220-273a-44a9-aafc-3cadcc0d5950 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.589854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee490c4e-1711-4b27-831e-49801e467aa7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.599612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:27.634822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004cf59-67d0-434c-83c2-4bdad4b1a626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.635463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004cf59-67d0-434c-83c2-4bdad4b1a626 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.635742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004cf59-67d0-434c-83c2-4bdad4b1a626 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.636041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004cf59-67d0-434c-83c2-4bdad4b1a626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.647977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004cf59-67d0-434c-83c2-4bdad4b1a626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.649657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004cf59-67d0-434c-83c2-4bdad4b1a626 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c004cf59-67d0-434c-83c2-4bdad4b1a626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:27.666437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:27.667603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:27.668845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:27.668845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:27.668845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:27.670661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 298/593] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:27.671673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004cf59-67d0-434c-83c2-4bdad4b1a626 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.671805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78410220-273a-44a9-aafc-3cadcc0d5950 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.692874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:27.713344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72423-6735-4240-89e9-e67f876d4426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.713985 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72423-6735-4240-89e9-e67f876d4426 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.714269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72423-6735-4240-89e9-e67f876d4426 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.714541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b72423-6735-4240-89e9-e67f876d4426 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.723039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b72423-6735-4240-89e9-e67f876d4426 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.724766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b72423-6735-4240-89e9-e67f876d4426 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-90b72423-6735-4240-89e9-e67f876d4426 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:27.738487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:27.739630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:27.745672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 296/594] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:27.747086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72423-6735-4240-89e9-e67f876d4426 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.794661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004cf59-67d0-434c-83c2-4bdad4b1a626 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.832773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a34a34-2c47-463b-8537-39501a2dbe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.833682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a34a34-2c47-463b-8537-39501a2dbe87 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.834121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a34a34-2c47-463b-8537-39501a2dbe87 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.834663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a34a34-2c47-463b-8537-39501a2dbe87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.842491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a34a34-2c47-463b-8537-39501a2dbe87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.844060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a34a34-2c47-463b-8537-39501a2dbe87 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-41a34a34-2c47-463b-8537-39501a2dbe87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:27.856772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:27.857875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 299/595] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:27.863547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a34a34-2c47-463b-8537-39501a2dbe87 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.868781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b72423-6735-4240-89e9-e67f876d4426 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:27.872370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:27.908973 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b40107-f210-4e24-aaeb-9d62db197a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:27.909962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b40107-f210-4e24-aaeb-9d62db197a6a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:27.910408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b40107-f210-4e24-aaeb-9d62db197a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:27.910830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b40107-f210-4e24-aaeb-9d62db197a6a None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:27.911688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b40107-f210-4e24-aaeb-9d62db197a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:27.912270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b40107-f210-4e24-aaeb-9d62db197a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:27.913964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b40107-f210-4e24-aaeb-9d62db197a6a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:27.928195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 297/596] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:27.929537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b40107-f210-4e24-aaeb-9d62db197a6a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:27.950013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:27.978950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a34a34-2c47-463b-8537-39501a2dbe87 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:28.024747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76db844-f8a3-43ff-b165-4f596c876567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:28.025536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76db844-f8a3-43ff-b165-4f596c876567 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.025917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76db844-f8a3-43ff-b165-4f596c876567 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.026289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76db844-f8a3-43ff-b165-4f596c876567 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.027390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76db844-f8a3-43ff-b165-4f596c876567 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:28.028312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76db844-f8a3-43ff-b165-4f596c876567 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:28.030135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76db844-f8a3-43ff-b165-4f596c876567 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:28.052349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 300/597] 10.222.0.71 () {56 vars in 1137 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:28.053774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76db844-f8a3-43ff-b165-4f596c876567 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:28.057648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b40107-f210-4e24-aaeb-9d62db197a6a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:28.098456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11668ae0-b0e9-491a-9154-e4d7942052cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:28.099363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11668ae0-b0e9-491a-9154-e4d7942052cd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.099814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11668ae0-b0e9-491a-9154-e4d7942052cd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.100222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11668ae0-b0e9-491a-9154-e4d7942052cd None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.101090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11668ae0-b0e9-491a-9154-e4d7942052cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:28.101546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11668ae0-b0e9-491a-9154-e4d7942052cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:28.104623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11668ae0-b0e9-491a-9154-e4d7942052cd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:28.122827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 298/598] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:28.124065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11668ae0-b0e9-491a-9154-e4d7942052cd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:28.155618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:28.198436 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76db844-f8a3-43ff-b165-4f596c876567 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:28.243658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11668ae0-b0e9-491a-9154-e4d7942052cd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:28.246410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768a6cb-869f-477c-b0dd-a4706f436604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:28.247895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768a6cb-869f-477c-b0dd-a4706f436604 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.248377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768a6cb-869f-477c-b0dd-a4706f436604 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.252142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768a6cb-869f-477c-b0dd-a4706f436604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.256237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:28.263976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768a6cb-869f-477c-b0dd-a4706f436604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:28.265776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768a6cb-869f-477c-b0dd-a4706f436604 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5768a6cb-869f-477c-b0dd-a4706f436604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:28.283046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:28.284038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:28.284921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:28.284921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:28.284921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:28.284921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:28.284921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:28.284921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:28.284921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:28.284921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:28.285871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 301/599] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:28.286334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768a6cb-869f-477c-b0dd-a4706f436604 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:28.297952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473e99c-6a81-42b6-8db7-6285049e1c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:28.298924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473e99c-6a81-42b6-8db7-6285049e1c31 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.299428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473e99c-6a81-42b6-8db7-6285049e1c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.299967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473e99c-6a81-42b6-8db7-6285049e1c31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.313822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473e99c-6a81-42b6-8db7-6285049e1c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:28.315595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473e99c-6a81-42b6-8db7-6285049e1c31 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3473e99c-6a81-42b6-8db7-6285049e1c31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:28.332687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:28.333656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:28.334527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:28.334527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:28.334527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:28.336653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 299/600] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:28.338355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473e99c-6a81-42b6-8db7-6285049e1c31 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:28.419838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768a6cb-869f-477c-b0dd-a4706f436604 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:28.434906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:28.465641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eb73b-bbc7-4365-bda0-0310db621e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:28.466583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eb73b-bbc7-4365-bda0-0310db621e29 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.466863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eb73b-bbc7-4365-bda0-0310db621e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.467135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eb73b-bbc7-4365-bda0-0310db621e29 None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/5c84b440821446e3a7ce97b2c68f815e/roles/fcc43e29f89e4c6cb1690a36d91ec4c1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.467932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473e99c-6a81-42b6-8db7-6285049e1c31 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:28.483202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0eb73b-bbc7-4365-bda0-0310db621e29 None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=5c84b440821446e3a7ce97b2c68f815e, role_id=fcc43e29f89e4c6cb1690a36d91ec4c1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:28.484749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0eb73b-bbc7-4365-bda0-0310db621e29 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:28.503954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 302/601] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:07:28 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/5c84b440821446e3a7ce97b2c68f815e/roles/fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:28.504963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eb73b-bbc7-4365-bda0-0310db621e29 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:28.523121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c77bb3-e4e5-41e4-b9af-f3a8e053f2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:28.524326 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c77bb3-e4e5-41e4-b9af-f3a8e053f2be None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.526546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c77bb3-e4e5-41e4-b9af-f3a8e053f2be None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.527122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c77bb3-e4e5-41e4-b9af-f3a8e053f2be None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.528892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c77bb3-e4e5-41e4-b9af-f3a8e053f2be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:28.529786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c77bb3-e4e5-41e4-b9af-f3a8e053f2be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:28.533721 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c77bb3-e4e5-41e4-b9af-f3a8e053f2be None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:28.566425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 300/602] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:28.571176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c77bb3-e4e5-41e4-b9af-f3a8e053f2be None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:28.627604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:28.735426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eb73b-bbc7-4365-bda0-0310db621e29 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:28.761451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c77bb3-e4e5-41e4-b9af-f3a8e053f2be None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:28.784147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff254ff-0191-4c34-990b-089524d2e3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:28.785135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff254ff-0191-4c34-990b-089524d2e3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.785623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff254ff-0191-4c34-990b-089524d2e3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.786121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff254ff-0191-4c34-990b-089524d2e3c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.787600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff254ff-0191-4c34-990b-089524d2e3c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:28.788363 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff254ff-0191-4c34-990b-089524d2e3c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:28.790596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff254ff-0191-4c34-990b-089524d2e3c1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:28.801079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd87f9-ffd4-4ed9-af9d-4900e5262cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:28.801903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd87f9-ffd4-4ed9-af9d-4900e5262cfe None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.802256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd87f9-ffd4-4ed9-af9d-4900e5262cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.802644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd87f9-ffd4-4ed9-af9d-4900e5262cfe None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/cd66d5d9989449729cec7e60cdaf0949/roles/fcc43e29f89e4c6cb1690a36d91ec4c1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.807738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:28.810022 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 303/603] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:28.811688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb0a47-1129-43a8-b51b-c391d07e56e6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.812210 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb0a47-1129-43a8-b51b-c391d07e56e6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.812672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb0a47-1129-43a8-b51b-c391d07e56e6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.815584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 304/604] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:28.816666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb0a47-1129-43a8-b51b-c391d07e56e6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:28.823610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd87f9-ffd4-4ed9-af9d-4900e5262cfe None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=cd66d5d9989449729cec7e60cdaf0949, role_id=fcc43e29f89e4c6cb1690a36d91ec4c1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:28.825217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd87f9-ffd4-4ed9-af9d-4900e5262cfe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:28.844021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 301/605] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:07:28 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/cd66d5d9989449729cec7e60cdaf0949/roles/fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:28.845612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755798c-db6d-4190-b1a1-a2552cb9259b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.846011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755798c-db6d-4190-b1a1-a2552cb9259b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.846360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755798c-db6d-4190-b1a1-a2552cb9259b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:28.871113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5755798c-db6d-4190-b1a1-a2552cb9259b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:28.872607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5755798c-db6d-4190-b1a1-a2552cb9259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:28.950374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb0a47-1129-43a8-b51b-c391d07e56e6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:28.959828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:28.972014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 302/606] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:28.993739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b7e08-f5fb-49d6-8124-3ea743e83b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:28.994514 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b7e08-f5fb-49d6-8124-3ea743e83b35 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:28.994869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b7e08-f5fb-49d6-8124-3ea743e83b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:28.995269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b7e08-f5fb-49d6-8124-3ea743e83b35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:29.003541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b7e08-f5fb-49d6-8124-3ea743e83b35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:29.005271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b7e08-f5fb-49d6-8124-3ea743e83b35 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dc9b7e08-f5fb-49d6-8124-3ea743e83b35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:29.019074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:29.020065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:29.021570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:29.021570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:29.021570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:29.021570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:29.021570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:29.021570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:29.021570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:29.021570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:29.021570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 305/607] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:29.026668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755798c-db6d-4190-b1a1-a2552cb9259b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:29.178208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755798c-db6d-4190-b1a1-a2552cb9259b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:29.197669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:29.223283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd117df-3a36-47f6-b580-ec654de34833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:29.223921 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd117df-3a36-47f6-b580-ec654de34833 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:29.224197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd117df-3a36-47f6-b580-ec654de34833 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:29.224463 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd117df-3a36-47f6-b580-ec654de34833 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:29.225152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd117df-3a36-47f6-b580-ec654de34833 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:29.225451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd117df-3a36-47f6-b580-ec654de34833 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:29.226871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd117df-3a36-47f6-b580-ec654de34833 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:29.240521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 303/608] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:29.246906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b7e08-f5fb-49d6-8124-3ea743e83b35 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:29.280502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd117df-3a36-47f6-b580-ec654de34833 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:29.427121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b7e08-f5fb-49d6-8124-3ea743e83b35 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:29.465319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c37de7-cd26-4694-bea6-1b8dbe0517c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:29.465904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c37de7-cd26-4694-bea6-1b8dbe0517c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:29.466438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c37de7-cd26-4694-bea6-1b8dbe0517c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:29.466716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c37de7-cd26-4694-bea6-1b8dbe0517c1 None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/56a94ccca15c462c9e7a5e88b907f420/roles/fcc43e29f89e4c6cb1690a36d91ec4c1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:29.468428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd117df-3a36-47f6-b580-ec654de34833 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:29.488669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c37de7-cd26-4694-bea6-1b8dbe0517c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=56a94ccca15c462c9e7a5e88b907f420, role_id=fcc43e29f89e4c6cb1690a36d91ec4c1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:29.490202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c37de7-cd26-4694-bea6-1b8dbe0517c1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:29.515938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 306/609] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:07:29 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/56a94ccca15c462c9e7a5e88b907f420/roles/fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:29.557306 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adee04-3481-4cac-af9e-efbf6a885ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:29.557956 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adee04-3481-4cac-af9e-efbf6a885ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:29.558231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adee04-3481-4cac-af9e-efbf6a885ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:29.558498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8adee04-3481-4cac-af9e-efbf6a885ce0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:29.584927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8adee04-3481-4cac-af9e-efbf6a885ce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a8adee04-3481-4cac-af9e-efbf6a885ce0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:29.590596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:29.591793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:29.592631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 304/610] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:29.599603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c37de7-cd26-4694-bea6-1b8dbe0517c1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:29.633293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:29.736649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c37de7-cd26-4694-bea6-1b8dbe0517c1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:29.790303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7569a-112c-4061-8d57-8096d3bbe19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:29.791005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7569a-112c-4061-8d57-8096d3bbe19a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:29.791288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7569a-112c-4061-8d57-8096d3bbe19a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:29.791562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7569a-112c-4061-8d57-8096d3bbe19a None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:29.792264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7569a-112c-4061-8d57-8096d3bbe19a None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:29.792566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7569a-112c-4061-8d57-8096d3bbe19a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:29.793958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7569a-112c-4061-8d57-8096d3bbe19a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:29.803002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 307/611] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:29.808616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adee04-3481-4cac-af9e-efbf6a885ce0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:29.863032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:29.874454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed546f-3d92-4095-8d71-030caef67119 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:29.874772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed546f-3d92-4095-8d71-030caef67119 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:29.875001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed546f-3d92-4095-8d71-030caef67119 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:29.876652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 308/612] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:29.881694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cb086-7281-4423-87f4-2572e9a55de4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:29.881979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cb086-7281-4423-87f4-2572e9a55de4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:29.882252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32cb086-7281-4423-87f4-2572e9a55de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:29.914277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d32cb086-7281-4423-87f4-2572e9a55de4 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:29.915511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d32cb086-7281-4423-87f4-2572e9a55de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:29.969730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8adee04-3481-4cac-af9e-efbf6a885ce0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:30.038733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d2dc-ac05-4b59-b10c-3bdd1d8f5286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:30.039286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d2dc-ac05-4b59-b10c-3bdd1d8f5286 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:30.039569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d2dc-ac05-4b59-b10c-3bdd1d8f5286 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:30.039862 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d2dc-ac05-4b59-b10c-3bdd1d8f5286 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:30.041110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 309/613] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:07:30.051950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162d2dc-ac05-4b59-b10c-3bdd1d8f5286 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:30.053480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162d2dc-ac05-4b59-b10c-3bdd1d8f5286 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2162d2dc-ac05-4b59-b10c-3bdd1d8f5286 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:30.076917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:30.077923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:30.078786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:30.078786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:30.078786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:30.078786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 305/614] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:30.085133 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cb086-7281-4423-87f4-2572e9a55de4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:30.100105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:30.221209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32cb086-7281-4423-87f4-2572e9a55de4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:30.257341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3b98f-b149-4d17-ae91-601a8d07720b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:30.257953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3b98f-b149-4d17-ae91-601a8d07720b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:30.258233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3b98f-b149-4d17-ae91-601a8d07720b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:30.258506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3b98f-b149-4d17-ae91-601a8d07720b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:30.259156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3b98f-b149-4d17-ae91-601a8d07720b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:30.259458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3b98f-b149-4d17-ae91-601a8d07720b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:30.261459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3b98f-b149-4d17-ae91-601a8d07720b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:30.273949 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 310/615] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:30.280897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d2dc-ac05-4b59-b10c-3bdd1d8f5286 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:30.391281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3b98f-b149-4d17-ae91-601a8d07720b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:30.406266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:30.431201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d2dc-ac05-4b59-b10c-3bdd1d8f5286 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:30.469493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c681b-8394-47c1-b3d0-9ba6eba8abee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:30.470126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c681b-8394-47c1-b3d0-9ba6eba8abee None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:30.470413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c681b-8394-47c1-b3d0-9ba6eba8abee None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:30.470686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c681b-8394-47c1-b3d0-9ba6eba8abee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:30.479929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c681b-8394-47c1-b3d0-9ba6eba8abee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:30.481400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c681b-8394-47c1-b3d0-9ba6eba8abee None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-bc1c681b-8394-47c1-b3d0-9ba6eba8abee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:30.494015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:30.495073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:30.496193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 306/616] 10.222.0.71 () {56 vars in 1093 bytes} [Wed Sep 21 12:07:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:30.500096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:30.504916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c681b-8394-47c1-b3d0-9ba6eba8abee None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:30.577977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3b98f-b149-4d17-ae91-601a8d07720b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:30.621063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c681b-8394-47c1-b3d0-9ba6eba8abee None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:30.660318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2142fbb-9da1-4829-847a-3965332ce17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:30.660859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2142fbb-9da1-4829-847a-3965332ce17b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:30.661189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2142fbb-9da1-4829-847a-3965332ce17b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:30.661409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2142fbb-9da1-4829-847a-3965332ce17b None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:30.662296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2142fbb-9da1-4829-847a-3965332ce17b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:30.662617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2142fbb-9da1-4829-847a-3965332ce17b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:30.664104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2142fbb-9da1-4829-847a-3965332ce17b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:30.668622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4124147-c178-464c-8b59-8e22b2a6e844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:30.669293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4124147-c178-464c-8b59-8e22b2a6e844 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:30.669573 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4124147-c178-464c-8b59-8e22b2a6e844 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:30.669849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4124147-c178-464c-8b59-8e22b2a6e844 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:30.690607 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4124147-c178-464c-8b59-8e22b2a6e844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b4124147-c178-464c-8b59-8e22b2a6e844 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:30.695905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:30.696812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:30.698181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 311/617] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:30.706381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 307/618] 10.222.0.71 () {56 vars in 1133 bytes} [Wed Sep 21 12:07:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:30.707455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4124147-c178-464c-8b59-8e22b2a6e844 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:30.712095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2142fbb-9da1-4829-847a-3965332ce17b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:30.726597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:30.866966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2142fbb-9da1-4829-847a-3965332ce17b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:30.907816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fdd99-b558-46fc-a756-d9545fa98acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:30.908733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fdd99-b558-46fc-a756-d9545fa98acd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:30.909106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fdd99-b558-46fc-a756-d9545fa98acd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:30.909625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fdd99-b558-46fc-a756-d9545fa98acd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:30.961787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4124147-c178-464c-8b59-8e22b2a6e844 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:30.981381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fdd99-b558-46fc-a756-d9545fa98acd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:30.983431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fdd99-b558-46fc-a756-d9545fa98acd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:30.996238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:31.068146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:31.083599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62acb-9ce1-4f23-9e26-e56edc8a6328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:31.084237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62acb-9ce1-4f23-9e26-e56edc8a6328 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.084511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62acb-9ce1-4f23-9e26-e56edc8a6328 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.084779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc62acb-9ce1-4f23-9e26-e56edc8a6328 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.085438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62acb-9ce1-4f23-9e26-e56edc8a6328 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:31.085737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62acb-9ce1-4f23-9e26-e56edc8a6328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:31.087104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc62acb-9ce1-4f23-9e26-e56edc8a6328 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:31.116178 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 312/619] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:31.117124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62acb-9ce1-4f23-9e26-e56edc8a6328 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:31.117788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 308/620] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:31.123902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fdd99-b558-46fc-a756-d9545fa98acd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:31.273629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc62acb-9ce1-4f23-9e26-e56edc8a6328 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:31.311947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afa2f8-a694-497d-90ce-f73c4a6085ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:31.312595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afa2f8-a694-497d-90ce-f73c4a6085ae None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.312914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afa2f8-a694-497d-90ce-f73c4a6085ae None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.313184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afa2f8-a694-497d-90ce-f73c4a6085ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.325900 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00afa2f8-a694-497d-90ce-f73c4a6085ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:31.328327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00afa2f8-a694-497d-90ce-f73c4a6085ae None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-00afa2f8-a694-497d-90ce-f73c4a6085ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:31.348662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:31.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:31.350561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:31.350561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:31.350561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:31.355556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 313/621] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:31.363591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afa2f8-a694-497d-90ce-f73c4a6085ae None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:31.383945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fdd99-b558-46fc-a756-d9545fa98acd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:31.393034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:31.428756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:31.432433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702bab5-e75e-4954-96d1-8315e40a1811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:31.433738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702bab5-e75e-4954-96d1-8315e40a1811 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.434232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702bab5-e75e-4954-96d1-8315e40a1811 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.434706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702bab5-e75e-4954-96d1-8315e40a1811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.446133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702bab5-e75e-4954-96d1-8315e40a1811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:31.447926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702bab5-e75e-4954-96d1-8315e40a1811 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1702bab5-e75e-4954-96d1-8315e40a1811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:31.465617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:31.466912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:31.467854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:31.467854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:31.467854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:31.469606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 309/622] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:31.471464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4913c52-e005-4bb4-9771-fb732d4c323f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.472249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4913c52-e005-4bb4-9771-fb732d4c323f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.473006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4913c52-e005-4bb4-9771-fb732d4c323f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.479676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 310/623] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:31.480857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4913c52-e005-4bb4-9771-fb732d4c323f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:31.515201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afa2f8-a694-497d-90ce-f73c4a6085ae None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:31.557340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31006ff9-7164-41b8-9362-c8ae749f8bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:31.557861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31006ff9-7164-41b8-9362-c8ae749f8bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.558136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31006ff9-7164-41b8-9362-c8ae749f8bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.558403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31006ff9-7164-41b8-9362-c8ae749f8bb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.559041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31006ff9-7164-41b8-9362-c8ae749f8bb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:31.559343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31006ff9-7164-41b8-9362-c8ae749f8bb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:31.560741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31006ff9-7164-41b8-9362-c8ae749f8bb7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:31.573064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 314/624] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:31.574187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3345a7-0fc3-49ad-96ef-92a6fecee06e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.574469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3345a7-0fc3-49ad-96ef-92a6fecee06e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.574737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3345a7-0fc3-49ad-96ef-92a6fecee06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.593523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cf3345a7-0fc3-49ad-96ef-92a6fecee06e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:31.594723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cf3345a7-0fc3-49ad-96ef-92a6fecee06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:31.604196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4913c52-e005-4bb4-9771-fb732d4c323f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:31.645265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1d1b9-43a3-4acc-b3c4-3b612edccaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:31.645931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1d1b9-43a3-4acc-b3c4-3b612edccaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.646321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1d1b9-43a3-4acc-b3c4-3b612edccaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.646427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1d1b9-43a3-4acc-b3c4-3b612edccaa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.647039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1d1b9-43a3-4acc-b3c4-3b612edccaa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:31.647339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1d1b9-43a3-4acc-b3c4-3b612edccaa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:31.648748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1d1b9-43a3-4acc-b3c4-3b612edccaa6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:31.669773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 315/625] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:31.669773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47154c-5755-4737-9fd8-4e8fbc13b5e0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.669773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47154c-5755-4737-9fd8-4e8fbc13b5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.669773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47154c-5755-4737-9fd8-4e8fbc13b5e0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.670209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 311/626] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:31.670209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1d1b9-43a3-4acc-b3c4-3b612edccaa6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:31.672509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 316/627] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:31.674633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47154c-5755-4737-9fd8-4e8fbc13b5e0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:31.693290 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:31.715080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:31.786656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1d1b9-43a3-4acc-b3c4-3b612edccaa6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:31.813104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47154c-5755-4737-9fd8-4e8fbc13b5e0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:31.830204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28d3b8-2b95-4c6f-8b93-6700598cb774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:31.831304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28d3b8-2b95-4c6f-8b93-6700598cb774 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.831891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28d3b8-2b95-4c6f-8b93-6700598cb774 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.832337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28d3b8-2b95-4c6f-8b93-6700598cb774 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.845106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28d3b8-2b95-4c6f-8b93-6700598cb774 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:31.846702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28d3b8-2b95-4c6f-8b93-6700598cb774 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:31.855461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d483b-ecf0-4797-a757-28d4a1843554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:31.856348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d483b-ecf0-4797-a757-28d4a1843554 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.856819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d483b-ecf0-4797-a757-28d4a1843554 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.857262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d483b-ecf0-4797-a757-28d4a1843554 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0b28d3b8-2b95-4c6f-8b93-6700598cb774 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:31.860096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:31.861017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:31.861955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:31.863884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 312/628] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:31.865281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e3983-0f66-480c-a575-0bb6a2c7ae8f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:31.865749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e3983-0f66-480c-a575-0bb6a2c7ae8f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:31.868130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e3983-0f66-480c-a575-0bb6a2c7ae8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:31.871001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d483b-ecf0-4797-a757-28d4a1843554 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:31.872869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d483b-ecf0-4797-a757-28d4a1843554 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dc0d483b-ecf0-4797-a757-28d4a1843554 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:31.885942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:31.886875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:31.887878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:31.887878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:31.887878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:31.887878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:31.887878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:31.887878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:31.887878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:31.887878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:31.893756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-778e3983-0f66-480c-a575-0bb6a2c7ae8f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:31.893756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-778e3983-0f66-480c-a575-0bb6a2c7ae8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:31.894003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 317/629] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:31.894568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d483b-ecf0-4797-a757-28d4a1843554 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:31.983783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 313/630] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:31.983783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e3983-0f66-480c-a575-0bb6a2c7ae8f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:31.995890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:32.010010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:32.024431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d483b-ecf0-4797-a757-28d4a1843554 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.067465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90feb1a-817b-4c8c-a255-bf085388b821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.068110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90feb1a-817b-4c8c-a255-bf085388b821 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.068387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90feb1a-817b-4c8c-a255-bf085388b821 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.068662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90feb1a-817b-4c8c-a255-bf085388b821 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.069395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90feb1a-817b-4c8c-a255-bf085388b821 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:32.069862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90feb1a-817b-4c8c-a255-bf085388b821 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:32.072401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90feb1a-817b-4c8c-a255-bf085388b821 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:32.095679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 318/631] 10.222.0.71 () {56 vars in 1135 bytes} [Wed Sep 21 12:07:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:32.096534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90feb1a-817b-4c8c-a255-bf085388b821 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:32.112613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e3983-0f66-480c-a575-0bb6a2c7ae8f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.154035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313cfa1-dd58-404e-8366-c4894188e395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.155145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313cfa1-dd58-404e-8366-c4894188e395 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.155566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313cfa1-dd58-404e-8366-c4894188e395 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.156133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313cfa1-dd58-404e-8366-c4894188e395 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.157476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313cfa1-dd58-404e-8366-c4894188e395 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:32.158042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313cfa1-dd58-404e-8366-c4894188e395 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:32.160099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8313cfa1-dd58-404e-8366-c4894188e395 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:32.176085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 314/632] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:32.177898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a6908-47c5-42ab-a15d-1f3ea6a14975 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.178313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a6908-47c5-42ab-a15d-1f3ea6a14975 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.179048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a6908-47c5-42ab-a15d-1f3ea6a14975 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.180957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 315/633] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:32.182372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a6908-47c5-42ab-a15d-1f3ea6a14975 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:32.205948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90feb1a-817b-4c8c-a255-bf085388b821 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.243047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ce988-d7e6-4905-b7bd-bd343029e9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.243805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ce988-d7e6-4905-b7bd-bd343029e9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.244682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ce988-d7e6-4905-b7bd-bd343029e9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.244955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ce988-d7e6-4905-b7bd-bd343029e9d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.255387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:32.256557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ce988-d7e6-4905-b7bd-bd343029e9d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-432ce988-d7e6-4905-b7bd-bd343029e9d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:32.261906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:32.262858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:32.263700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 319/634] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:32.264471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ce988-d7e6-4905-b7bd-bd343029e9d7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:32.280455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:32.291299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a6908-47c5-42ab-a15d-1f3ea6a14975 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.325541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b85bd-ce57-4a61-8a65-2354411cd58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.326094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b85bd-ce57-4a61-8a65-2354411cd58e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.326377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b85bd-ce57-4a61-8a65-2354411cd58e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.326647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b85bd-ce57-4a61-8a65-2354411cd58e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.337803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b85bd-ce57-4a61-8a65-2354411cd58e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:32.339392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b85bd-ce57-4a61-8a65-2354411cd58e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-769b85bd-ce57-4a61-8a65-2354411cd58e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:32.353917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:32.354858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:32.355740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:32.355740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:32.355740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:32.355740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 316/635] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:32.356630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc56d49-6741-45bf-9d2a-56d17d5971e1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.356894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc56d49-6741-45bf-9d2a-56d17d5971e1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.357159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc56d49-6741-45bf-9d2a-56d17d5971e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.373589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ce988-d7e6-4905-b7bd-bd343029e9d7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.376143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2cc56d49-6741-45bf-9d2a-56d17d5971e1 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:32.377334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2cc56d49-6741-45bf-9d2a-56d17d5971e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:32.409295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2381f-bbd6-4206-b042-dd3365b1eb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.409907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2381f-bbd6-4206-b042-dd3365b1eb7e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.410183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2381f-bbd6-4206-b042-dd3365b1eb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.410459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2381f-bbd6-4206-b042-dd3365b1eb7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.411172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2381f-bbd6-4206-b042-dd3365b1eb7e None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=463487449c1d4aada1b5f279c959b0e6) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:32.411481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2381f-bbd6-4206-b042-dd3365b1eb7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:32.412953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2381f-bbd6-4206-b042-dd3365b1eb7e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:32.418621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 320/636] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:32 2022] GET /identity/v3/role_assignments?user.id=463487449c1d4aada1b5f279c959b0e6&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:32.420711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2381f-bbd6-4206-b042-dd3365b1eb7e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:32.455785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 317/637] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:32.455785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc56d49-6741-45bf-9d2a-56d17d5971e1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:32.520102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:32.534697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:32.566600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2381f-bbd6-4206-b042-dd3365b1eb7e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.596726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc56d49-6741-45bf-9d2a-56d17d5971e1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.607566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b4634-10b4-42d4-92e6-65582c391dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.608231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b4634-10b4-42d4-92e6-65582c391dbb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.608507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b4634-10b4-42d4-92e6-65582c391dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.608777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b4634-10b4-42d4-92e6-65582c391dbb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.621582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b4634-10b4-42d4-92e6-65582c391dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3d3b4634-10b4-42d4-92e6-65582c391dbb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:32.627016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:32.628033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 321/638] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:32.629277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b4634-10b4-42d4-92e6-65582c391dbb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:32.644512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0437-cd4c-4d43-81ca-7b833638aa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.645265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0437-cd4c-4d43-81ca-7b833638aa2b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.645701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0437-cd4c-4d43-81ca-7b833638aa2b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.646227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0437-cd4c-4d43-81ca-7b833638aa2b None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.646656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f0437-cd4c-4d43-81ca-7b833638aa2b None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:32.646955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f0437-cd4c-4d43-81ca-7b833638aa2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:32.649047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f0437-cd4c-4d43-81ca-7b833638aa2b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:32.659601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 318/639] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:32.660417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0437-cd4c-4d43-81ca-7b833638aa2b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:32.757236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b4634-10b4-42d4-92e6-65582c391dbb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.786268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0437-cd4c-4d43-81ca-7b833638aa2b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.798583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865eea2-15d9-4ac5-acac-3b7ef87da85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.799190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865eea2-15d9-4ac5-acac-3b7ef87da85b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.799458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865eea2-15d9-4ac5-acac-3b7ef87da85b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.799731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865eea2-15d9-4ac5-acac-3b7ef87da85b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.800392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865eea2-15d9-4ac5-acac-3b7ef87da85b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:32.800694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865eea2-15d9-4ac5-acac-3b7ef87da85b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:32.801852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:32.802112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865eea2-15d9-4ac5-acac-3b7ef87da85b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:32.805466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:32.822683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 322/640] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:32.822683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865eea2-15d9-4ac5-acac-3b7ef87da85b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:32.830335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d137127-2728-4d42-97df-bdd758f664af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.830335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d137127-2728-4d42-97df-bdd758f664af None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.830335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d137127-2728-4d42-97df-bdd758f664af None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.830335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d137127-2728-4d42-97df-bdd758f664af None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.830694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d137127-2728-4d42-97df-bdd758f664af None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:32.830694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d137127-2728-4d42-97df-bdd758f664af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:32.832166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d137127-2728-4d42-97df-bdd758f664af None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:32.841997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 319/641] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:32.847836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d137127-2728-4d42-97df-bdd758f664af None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:32.947429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865eea2-15d9-4ac5-acac-3b7ef87da85b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.964151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d137127-2728-4d42-97df-bdd758f664af None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:32.983026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c60172-e22b-43b9-972d-c4d878b9acea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:32.983662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c60172-e22b-43b9-972d-c4d878b9acea None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:32.983966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c60172-e22b-43b9-972d-c4d878b9acea None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:32.984245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c60172-e22b-43b9-972d-c4d878b9acea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:32.995785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c60172-e22b-43b9-972d-c4d878b9acea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:32.998023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c60172-e22b-43b9-972d-c4d878b9acea None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.001526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9d6a0-d25d-47c3-957c-a08371b9a86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.002155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9d6a0-d25d-47c3-957c-a08371b9a86c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.002426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9d6a0-d25d-47c3-957c-a08371b9a86c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.002694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9d6a0-d25d-47c3-957c-a08371b9a86c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.013608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9d6a0-d25d-47c3-957c-a08371b9a86c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e3c60172-e22b-43b9-972d-c4d878b9acea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:33.014913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:33.016050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:33.016942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:33.016942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:33.016942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-dcf9d6a0-d25d-47c3-957c-a08371b9a86c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:33.018842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:33.019797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:33.021126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 320/642] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:33.022949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9d6a0-d25d-47c3-957c-a08371b9a86c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.023398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 323/643] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:33.024677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c60172-e22b-43b9-972d-c4d878b9acea None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.059452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:33.077223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:33.155349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9d6a0-d25d-47c3-957c-a08371b9a86c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.161499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c60172-e22b-43b9-972d-c4d878b9acea None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.201090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13084599-b0ab-4291-97c8-4d2826e7d69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.201726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13084599-b0ab-4291-97c8-4d2826e7d69b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.202010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13084599-b0ab-4291-97c8-4d2826e7d69b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.202282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13084599-b0ab-4291-97c8-4d2826e7d69b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.209275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13084599-b0ab-4291-97c8-4d2826e7d69b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.210739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13084599-b0ab-4291-97c8-4d2826e7d69b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.212968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca6dd2-b210-4315-b292-b1c155a72f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.213848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca6dd2-b210-4315-b292-b1c155a72f8c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.214266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca6dd2-b210-4315-b292-b1c155a72f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.214673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca6dd2-b210-4315-b292-b1c155a72f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-13084599-b0ab-4291-97c8-4d2826e7d69b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:33.223388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:33.224548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 321/644] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:33.225573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13084599-b0ab-4291-97c8-4d2826e7d69b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.230574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca6dd2-b210-4315-b292-b1c155a72f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.233327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca6dd2-b210-4315-b292-b1c155a72f8c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-79ca6dd2-b210-4315-b292-b1c155a72f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:33.252382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:33.253556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:33.254506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:33.254506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:33.254506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:33.254506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 324/645] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:33.255407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca6dd2-b210-4315-b292-b1c155a72f8c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.338061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:33.348328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13084599-b0ab-4291-97c8-4d2826e7d69b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.362589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:33.366673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca6dd2-b210-4315-b292-b1c155a72f8c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.389050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e645e9-3ddf-4f3f-8288-81539d940c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.389666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e645e9-3ddf-4f3f-8288-81539d940c85 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.389944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e645e9-3ddf-4f3f-8288-81539d940c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.390216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e645e9-3ddf-4f3f-8288-81539d940c85 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.391027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e645e9-3ddf-4f3f-8288-81539d940c85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:33.391672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e645e9-3ddf-4f3f-8288-81539d940c85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.393373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e645e9-3ddf-4f3f-8288-81539d940c85 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.402209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f548c5-0678-451b-b0ea-e71e1238f429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.402816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f548c5-0678-451b-b0ea-e71e1238f429 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.403089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f548c5-0678-451b-b0ea-e71e1238f429 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.403355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f548c5-0678-451b-b0ea-e71e1238f429 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.404045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f548c5-0678-451b-b0ea-e71e1238f429 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:33.404344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f548c5-0678-451b-b0ea-e71e1238f429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.405743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f548c5-0678-451b-b0ea-e71e1238f429 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.412033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 322/646] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:33.413077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e645e9-3ddf-4f3f-8288-81539d940c85 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.414037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 325/647] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:33.414843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f548c5-0678-451b-b0ea-e71e1238f429 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.526337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f548c5-0678-451b-b0ea-e71e1238f429 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.543970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e645e9-3ddf-4f3f-8288-81539d940c85 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.570285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5b90a-61ad-4888-a28e-3e3c9eb63b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.571173 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5b90a-61ad-4888-a28e-3e3c9eb63b53 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.571456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5b90a-61ad-4888-a28e-3e3c9eb63b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.572454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5b90a-61ad-4888-a28e-3e3c9eb63b53 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.573112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5b90a-61ad-4888-a28e-3e3c9eb63b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:33.573425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5b90a-61ad-4888-a28e-3e3c9eb63b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.574807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5b90a-61ad-4888-a28e-3e3c9eb63b53 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.581201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6acd75-aa11-4963-98a1-393038c98754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.581799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6acd75-aa11-4963-98a1-393038c98754 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.582076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6acd75-aa11-4963-98a1-393038c98754 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.582344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6acd75-aa11-4963-98a1-393038c98754 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.583020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6acd75-aa11-4963-98a1-393038c98754 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:33.583318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6acd75-aa11-4963-98a1-393038c98754 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.584745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6acd75-aa11-4963-98a1-393038c98754 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.592568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 326/648] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:33.592568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5b90a-61ad-4888-a28e-3e3c9eb63b53 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.609530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 323/649] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:33.609530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6acd75-aa11-4963-98a1-393038c98754 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.619847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:33.622001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:33.710650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5b90a-61ad-4888-a28e-3e3c9eb63b53 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.728079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6acd75-aa11-4963-98a1-393038c98754 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.752002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf20261-4284-47b0-bcae-af587369cc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.752638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf20261-4284-47b0-bcae-af587369cc20 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.752920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf20261-4284-47b0-bcae-af587369cc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.753193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf20261-4284-47b0-bcae-af587369cc20 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.761766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf20261-4284-47b0-bcae-af587369cc20 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.763497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf20261-4284-47b0-bcae-af587369cc20 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.772880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7707c-52b2-476f-90a7-8f9f222dd5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.773593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7707c-52b2-476f-90a7-8f9f222dd5aa None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.773939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7707c-52b2-476f-90a7-8f9f222dd5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.774277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7707c-52b2-476f-90a7-8f9f222dd5aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9cf20261-4284-47b0-bcae-af587369cc20 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:33.777140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:33.778019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 327/650] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:33.783683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf20261-4284-47b0-bcae-af587369cc20 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.786636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7707c-52b2-476f-90a7-8f9f222dd5aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.788111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7707c-52b2-476f-90a7-8f9f222dd5aa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6ad7707c-52b2-476f-90a7-8f9f222dd5aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:33.803115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:33.804116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:33.805025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:33.805025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:33.805025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:33.805025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 324/651] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:33.805025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7707c-52b2-476f-90a7-8f9f222dd5aa None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.881256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:33.885525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:33.907301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf20261-4284-47b0-bcae-af587369cc20 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.924791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7707c-52b2-476f-90a7-8f9f222dd5aa None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:33.946567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2041a5-e9f5-44e6-89d5-e09788407028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.947901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2041a5-e9f5-44e6-89d5-e09788407028 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.948134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2041a5-e9f5-44e6-89d5-e09788407028 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.948442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2041a5-e9f5-44e6-89d5-e09788407028 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.956288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2041a5-e9f5-44e6-89d5-e09788407028 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.957730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2041a5-e9f5-44e6-89d5-e09788407028 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:33.962328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4762c97-a156-4ffb-991e-3bb477f58467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:33.962959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4762c97-a156-4ffb-991e-3bb477f58467 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:33.963270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4762c97-a156-4ffb-991e-3bb477f58467 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:33.963593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4762c97-a156-4ffb-991e-3bb477f58467 None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/e0bf75c730c946bc9168dd418e7af637/roles/fcc43e29f89e4c6cb1690a36d91ec4c1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6b2041a5-e9f5-44e6-89d5-e09788407028 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:33.969227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:33.970235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:33.971177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 328/652] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:33.972379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2041a5-e9f5-44e6-89d5-e09788407028 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:33.983622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4762c97-a156-4ffb-991e-3bb477f58467 None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=e0bf75c730c946bc9168dd418e7af637, role_id=fcc43e29f89e4c6cb1690a36d91ec4c1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:33.985098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4762c97-a156-4ffb-991e-3bb477f58467 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:34.003832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 325/653] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:07:33 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/e0bf75c730c946bc9168dd418e7af637/roles/fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:34.004717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4762c97-a156-4ffb-991e-3bb477f58467 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:34.099122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2041a5-e9f5-44e6-89d5-e09788407028 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:34.136409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4762c97-a156-4ffb-991e-3bb477f58467 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:34.151974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6014954-91ec-4bca-8733-1bc057780d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:34.152640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6014954-91ec-4bca-8733-1bc057780d05 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:34.152928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6014954-91ec-4bca-8733-1bc057780d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:34.153205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6014954-91ec-4bca-8733-1bc057780d05 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:34.154294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6014954-91ec-4bca-8733-1bc057780d05 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:34.154676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6014954-91ec-4bca-8733-1bc057780d05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:34.156397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6014954-91ec-4bca-8733-1bc057780d05 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:34.170920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:34.173311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:34.182890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 329/654] 10.222.0.71 () {56 vars in 1137 bytes} [Wed Sep 21 12:07:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:34.184169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6014954-91ec-4bca-8733-1bc057780d05 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:34.199139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a9273-6936-4a78-9eef-a98037740b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:34.199881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a9273-6936-4a78-9eef-a98037740b15 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:34.200344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a9273-6936-4a78-9eef-a98037740b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:34.200699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a9273-6936-4a78-9eef-a98037740b15 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:34.201387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a9273-6936-4a78-9eef-a98037740b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:34.201793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a9273-6936-4a78-9eef-a98037740b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:34.203416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a9273-6936-4a78-9eef-a98037740b15 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:34.217585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 326/655] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:34.218275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a9273-6936-4a78-9eef-a98037740b15 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:34.307560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6014954-91ec-4bca-8733-1bc057780d05 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:34.330592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a9273-6936-4a78-9eef-a98037740b15 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:34.359536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602a32-5888-436d-8647-7ff0a4ffc5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:34.360172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602a32-5888-436d-8647-7ff0a4ffc5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:34.360446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602a32-5888-436d-8647-7ff0a4ffc5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:34.360713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0602a32-5888-436d-8647-7ff0a4ffc5c8 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:34.361444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0602a32-5888-436d-8647-7ff0a4ffc5c8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:34.361909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0602a32-5888-436d-8647-7ff0a4ffc5c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:34.364241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0602a32-5888-436d-8647-7ff0a4ffc5c8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:34.384152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4390ece-36ee-4beb-83d6-f0dc41ece788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:34.385662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 330/656] 10.222.0.71 () {56 vars in 1139 bytes} [Wed Sep 21 12:07:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:34.385972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390ece-36ee-4beb-83d6-f0dc41ece788 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:34.386553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390ece-36ee-4beb-83d6-f0dc41ece788 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:34.386661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602a32-5888-436d-8647-7ff0a4ffc5c8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:34.387221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390ece-36ee-4beb-83d6-f0dc41ece788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:34.445860 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:34.452341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:34.484872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4390ece-36ee-4beb-83d6-f0dc41ece788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:34.487119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4390ece-36ee-4beb-83d6-f0dc41ece788 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:34.496534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0602a32-5888-436d-8647-7ff0a4ffc5c8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:34.535256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0bfc3-b7f9-48b3-b3f9-ea8cfe7d234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:34.535914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0bfc3-b7f9-48b3-b3f9-ea8cfe7d234f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:34.536190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0bfc3-b7f9-48b3-b3f9-ea8cfe7d234f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:34.536457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0bfc3-b7f9-48b3-b3f9-ea8cfe7d234f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:34.544527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0bfc3-b7f9-48b3-b3f9-ea8cfe7d234f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:34.545871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0bfc3-b7f9-48b3-b3f9-ea8cfe7d234f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7ee0bfc3-b7f9-48b3-b3f9-ea8cfe7d234f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:34.559223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:34.560277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:34.561227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 331/657] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:34.561842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0bfc3-b7f9-48b3-b3f9-ea8cfe7d234f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:34.628297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 327/658] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:34.630038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4390ece-36ee-4beb-83d6-f0dc41ece788 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:34.692410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0bfc3-b7f9-48b3-b3f9-ea8cfe7d234f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:34.716782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:34.720063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:34.739597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e42e5-d3bd-4738-afea-11f7b6bc5173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:34.740174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e42e5-d3bd-4738-afea-11f7b6bc5173 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:34.740454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e42e5-d3bd-4738-afea-11f7b6bc5173 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:34.740731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e42e5-d3bd-4738-afea-11f7b6bc5173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:34.753373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4390ece-36ee-4beb-83d6-f0dc41ece788 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:34.791557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e0968-085d-4899-979f-03931b5b8849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:34.792297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e0968-085d-4899-979f-03931b5b8849 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:34.792484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e0968-085d-4899-979f-03931b5b8849 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:34.792807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e0968-085d-4899-979f-03931b5b8849 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:34.793539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e0968-085d-4899-979f-03931b5b8849 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:34.793848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e0968-085d-4899-979f-03931b5b8849 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:34.795361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e0968-085d-4899-979f-03931b5b8849 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:34.818809 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 328/659] 10.222.0.71 () {56 vars in 1137 bytes} [Wed Sep 21 12:07:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:34.818809 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e0968-085d-4899-979f-03931b5b8849 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:34.827535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e42e5-d3bd-4738-afea-11f7b6bc5173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:34.829157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e42e5-d3bd-4738-afea-11f7b6bc5173 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:34.930659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e0968-085d-4899-979f-03931b5b8849 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:34.940653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 332/660] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:34.940653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b60b-0dfb-49bb-972f-44fd544039dc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:34.940653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b60b-0dfb-49bb-972f-44fd544039dc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:34.940653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b60b-0dfb-49bb-972f-44fd544039dc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:34.940653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 333/661] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:34.940653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b60b-0dfb-49bb-972f-44fd544039dc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:34.958673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:34.973687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3fb2f-4739-4fed-8b34-2ca501322ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:34.974321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3fb2f-4739-4fed-8b34-2ca501322ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:34.974605 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3fb2f-4739-4fed-8b34-2ca501322ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:34.974874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3fb2f-4739-4fed-8b34-2ca501322ef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:34.977051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:34.990399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3fb2f-4739-4fed-8b34-2ca501322ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:34.992076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3fb2f-4739-4fed-8b34-2ca501322ef7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a1f3fb2f-4739-4fed-8b34-2ca501322ef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:35.009991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:35.011146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:35.012106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:35.012106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:35.012106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:35.012106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 329/662] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:35.012106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3fb2f-4739-4fed-8b34-2ca501322ef7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:35.057796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b60b-0dfb-49bb-972f-44fd544039dc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:35.092917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38bda9-0980-4a7d-8def-ca0dd7081e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:35.093523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38bda9-0980-4a7d-8def-ca0dd7081e9f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:35.093807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38bda9-0980-4a7d-8def-ca0dd7081e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:35.094132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38bda9-0980-4a7d-8def-ca0dd7081e9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:35.128826 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3fb2f-4739-4fed-8b34-2ca501322ef7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:35.162491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57f348-6797-4f4c-8977-32155ad334e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:35.163126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57f348-6797-4f4c-8977-32155ad334e4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:35.163407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57f348-6797-4f4c-8977-32155ad334e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:35.163681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57f348-6797-4f4c-8977-32155ad334e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:35.164600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38bda9-0980-4a7d-8def-ca0dd7081e9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:35.166577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38bda9-0980-4a7d-8def-ca0dd7081e9f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:35.175244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa57f348-6797-4f4c-8977-32155ad334e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:35.176719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa57f348-6797-4f4c-8977-32155ad334e4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:35.184427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-aa57f348-6797-4f4c-8977-32155ad334e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:35.193496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:35.194451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:35.195348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:35.195348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:35.195348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:35.195348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 330/663] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:35.196161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8081f-dbb0-4c5e-9849-8c9aebb05cf6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:35.196161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8081f-dbb0-4c5e-9849-8c9aebb05cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:35.196305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8081f-dbb0-4c5e-9849-8c9aebb05cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:35.215571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a5b8081f-dbb0-4c5e-9849-8c9aebb05cf6 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:35.217045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a5b8081f-dbb0-4c5e-9849-8c9aebb05cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:35.250310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:35.282844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 334/664] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 12:07:35.283040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38bda9-0980-4a7d-8def-ca0dd7081e9f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:35.303868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 331/665] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:35.303868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8081f-dbb0-4c5e-9849-8c9aebb05cf6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:35.420521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8081f-dbb0-4c5e-9849-8c9aebb05cf6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:35.433898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38bda9-0980-4a7d-8def-ca0dd7081e9f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:35.457227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:35.458565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7521e-9603-4a36-b0f2-6acd622e5d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:35.459286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7521e-9603-4a36-b0f2-6acd622e5d54 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:35.459568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7521e-9603-4a36-b0f2-6acd622e5d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:35.459880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7521e-9603-4a36-b0f2-6acd622e5d54 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:35.460665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7521e-9603-4a36-b0f2-6acd622e5d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:35.461075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7521e-9603-4a36-b0f2-6acd622e5d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:35.462755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc7521e-9603-4a36-b0f2-6acd622e5d54 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:35.479690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 332/666] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:35.480683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7521e-9603-4a36-b0f2-6acd622e5d54 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:35.481590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec95ed-da9d-441c-8d57-81af5aa09cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:35.482447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec95ed-da9d-441c-8d57-81af5aa09cea None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:35.482904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec95ed-da9d-441c-8d57-81af5aa09cea None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:35.483620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec95ed-da9d-441c-8d57-81af5aa09cea None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:35.485447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec95ed-da9d-441c-8d57-81af5aa09cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:35.486001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec95ed-da9d-441c-8d57-81af5aa09cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:35.487610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec95ed-da9d-441c-8d57-81af5aa09cea None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:35.504267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 335/667] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:35.506008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec95ed-da9d-441c-8d57-81af5aa09cea None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:35.512430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:35.603728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc7521e-9603-4a36-b0f2-6acd622e5d54 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:35.620247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec95ed-da9d-441c-8d57-81af5aa09cea None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:35.642580 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08c79f-5c59-45cb-b120-26337261ef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:35.643343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08c79f-5c59-45cb-b120-26337261ef60 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:35.643713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08c79f-5c59-45cb-b120-26337261ef60 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:35.644093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08c79f-5c59-45cb-b120-26337261ef60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:35.655928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08c79f-5c59-45cb-b120-26337261ef60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:35.657568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd08c79f-5c59-45cb-b120-26337261ef60 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:35.657679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a0e54-94b2-42db-848b-18b520422962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:35.658337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a0e54-94b2-42db-848b-18b520422962 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:35.658685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a0e54-94b2-42db-848b-18b520422962 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:35.659023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a0e54-94b2-42db-848b-18b520422962 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:35.668737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a0e54-94b2-42db-848b-18b520422962 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:35.670243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a0e54-94b2-42db-848b-18b520422962 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-cd08c79f-5c59-45cb-b120-26337261ef60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:35.674141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:35.675295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:35.676676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:35.676676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:35.676676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:35.680150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 333/668] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:35.680851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08c79f-5c59-45cb-b120-26337261ef60 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9b2a0e54-94b2-42db-848b-18b520422962 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:35.683785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:35.684956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:35.685940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:35.685940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:35.685940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:35.685940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:35.685940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:35.685940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:35.685940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:35.685940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:35.686570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 336/669] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:35.687196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a0e54-94b2-42db-848b-18b520422962 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:35.733829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:35.778033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:35.799162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08c79f-5c59-45cb-b120-26337261ef60 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:35.809168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a0e54-94b2-42db-848b-18b520422962 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:35.834813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c77f2-46bb-43b8-b750-1c6123a8aa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:35.835447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c77f2-46bb-43b8-b750-1c6123a8aa5e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:35.835726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c77f2-46bb-43b8-b750-1c6123a8aa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:35.836024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c77f2-46bb-43b8-b750-1c6123a8aa5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:35.843898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c77f2-46bb-43b8-b750-1c6123a8aa5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:35.845361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c77f2-46bb-43b8-b750-1c6123a8aa5e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:35.846163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b69b1-e2bf-4183-824d-4281ccdce6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:35.846888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b69b1-e2bf-4183-824d-4281ccdce6fe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:35.847230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b69b1-e2bf-4183-824d-4281ccdce6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:35.847560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b69b1-e2bf-4183-824d-4281ccdce6fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-cf0c77f2-46bb-43b8-b750-1c6123a8aa5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:35.857332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:35.858412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:35.859507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:35.859507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:35.859507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:35.859507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:35.859507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:35.859507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:35.859507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:35.859507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:35.860454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 334/670] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:35.861281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c77f2-46bb-43b8-b750-1c6123a8aa5e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:35.865868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b69b1-e2bf-4183-824d-4281ccdce6fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-419b69b1-e2bf-4183-824d-4281ccdce6fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:35.871300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:35.875852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 337/671] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:35.876832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b69b1-e2bf-4183-824d-4281ccdce6fe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:35.975606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c77f2-46bb-43b8-b750-1c6123a8aa5e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:35.977367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:35.985443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b69b1-e2bf-4183-824d-4281ccdce6fe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.011608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008af8e-d956-4705-ab88-44df30c8321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.012244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008af8e-d956-4705-ab88-44df30c8321c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.012519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008af8e-d956-4705-ab88-44df30c8321c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.012785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008af8e-d956-4705-ab88-44df30c8321c None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.013416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008af8e-d956-4705-ab88-44df30c8321c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:36.013713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008af8e-d956-4705-ab88-44df30c8321c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.015120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008af8e-d956-4705-ab88-44df30c8321c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.023729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:36.026648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d949e0-1a42-43ff-a9f7-e7d633302972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.027304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d949e0-1a42-43ff-a9f7-e7d633302972 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.027650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d949e0-1a42-43ff-a9f7-e7d633302972 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.028012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d949e0-1a42-43ff-a9f7-e7d633302972 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.028856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d949e0-1a42-43ff-a9f7-e7d633302972 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:36.029148 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d949e0-1a42-43ff-a9f7-e7d633302972 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.030621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d949e0-1a42-43ff-a9f7-e7d633302972 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.032945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 335/672] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:36.033806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008af8e-d956-4705-ab88-44df30c8321c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.047987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 338/673] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:36.047987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d949e0-1a42-43ff-a9f7-e7d633302972 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.149779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008af8e-d956-4705-ab88-44df30c8321c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.155676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d949e0-1a42-43ff-a9f7-e7d633302972 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.188265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dc226-2a50-4edb-b9d4-1dbc4230657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.188879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dc226-2a50-4edb-b9d4-1dbc4230657f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.189157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dc226-2a50-4edb-b9d4-1dbc4230657f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.189427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dc226-2a50-4edb-b9d4-1dbc4230657f None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.190112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dc226-2a50-4edb-b9d4-1dbc4230657f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:36.190415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dc226-2a50-4edb-b9d4-1dbc4230657f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.191917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dc226-2a50-4edb-b9d4-1dbc4230657f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.196598 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8090bd-3d8e-4b58-8662-c8417bb2e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.197806 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8090bd-3d8e-4b58-8662-c8417bb2e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.198148 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8090bd-3d8e-4b58-8662-c8417bb2e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.198481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8090bd-3d8e-4b58-8662-c8417bb2e0a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.199552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8090bd-3d8e-4b58-8662-c8417bb2e0a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:36.199552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8090bd-3d8e-4b58-8662-c8417bb2e0a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.200872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8090bd-3d8e-4b58-8662-c8417bb2e0a1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.210156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 336/674] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:36.210156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dc226-2a50-4edb-b9d4-1dbc4230657f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.219545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 339/675] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:36.219545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8090bd-3d8e-4b58-8662-c8417bb2e0a1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.244508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:36.274502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:36.322821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dc226-2a50-4edb-b9d4-1dbc4230657f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.327903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8090bd-3d8e-4b58-8662-c8417bb2e0a1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.357932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405b768-a3c1-4374-b539-cc6af9fe6c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.358944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405b768-a3c1-4374-b539-cc6af9fe6c3f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.359235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405b768-a3c1-4374-b539-cc6af9fe6c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.359503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a405b768-a3c1-4374-b539-cc6af9fe6c3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.362113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c12d3c-d4a8-4d60-aea4-48b4eec7f439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.362713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c12d3c-d4a8-4d60-aea4-48b4eec7f439 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.362993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c12d3c-d4a8-4d60-aea4-48b4eec7f439 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.363262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c12d3c-d4a8-4d60-aea4-48b4eec7f439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.364410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c12d3c-d4a8-4d60-aea4-48b4eec7f439 None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=cd66d5d9989449729cec7e60cdaf0949) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:36.364410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c12d3c-d4a8-4d60-aea4-48b4eec7f439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.365808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c12d3c-d4a8-4d60-aea4-48b4eec7f439 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.366153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405b768-a3c1-4374-b539-cc6af9fe6c3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.367577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a405b768-a3c1-4374-b539-cc6af9fe6c3f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.371560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 340/676] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/role_assignments?user.id=cd66d5d9989449729cec7e60cdaf0949&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:36.372376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c12d3c-d4a8-4d60-aea4-48b4eec7f439 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a405b768-a3c1-4374-b539-cc6af9fe6c3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:36.378998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:36.380180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:36.381193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:36.381193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:36.381193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:36.381193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:36.381193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:36.381193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:36.381193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:36.381193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:36.381857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 337/677] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:36.383042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405b768-a3c1-4374-b539-cc6af9fe6c3f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.475494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:36.497710 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c12d3c-d4a8-4d60-aea4-48b4eec7f439 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.500651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a405b768-a3c1-4374-b539-cc6af9fe6c3f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.524660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:36.536760 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e23071-7155-4b1c-b196-4925f7ab26c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.537613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e23071-7155-4b1c-b196-4925f7ab26c7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.537732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e23071-7155-4b1c-b196-4925f7ab26c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.537921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e23071-7155-4b1c-b196-4925f7ab26c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.538654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e23071-7155-4b1c-b196-4925f7ab26c7 None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=5c84b440821446e3a7ce97b2c68f815e) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:36.538964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e23071-7155-4b1c-b196-4925f7ab26c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.540446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e23071-7155-4b1c-b196-4925f7ab26c7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.542910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf735a-8c90-4e7f-8ff2-0592f5e6636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.543780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf735a-8c90-4e7f-8ff2-0592f5e6636d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.544562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf735a-8c90-4e7f-8ff2-0592f5e6636d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.544562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf735a-8c90-4e7f-8ff2-0592f5e6636d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.547019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 341/678] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/role_assignments?user.id=5c84b440821446e3a7ce97b2c68f815e&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:36.548429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecec891-140c-45b9-9ed8-f7ee913057de None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.548851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecec891-140c-45b9-9ed8-f7ee913057de None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.549189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecec891-140c-45b9-9ed8-f7ee913057de None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.555554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 342/679] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:36.555554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecec891-140c-45b9-9ed8-f7ee913057de None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.571287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf735a-8c90-4e7f-8ff2-0592f5e6636d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.573712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf735a-8c90-4e7f-8ff2-0592f5e6636d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-98cf735a-8c90-4e7f-8ff2-0592f5e6636d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:36.603786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:36.604799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:36.605698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:36.605698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:36.605698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:36.605698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 338/680] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:36.605929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77c553-6da9-40ae-ae31-b3218b4e745c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.606779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77c553-6da9-40ae-ae31-b3218b4e745c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.606779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77c553-6da9-40ae-ae31-b3218b4e745c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.630347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0c77c553-6da9-40ae-ae31-b3218b4e745c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:36.631568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0c77c553-6da9-40ae-ae31-b3218b4e745c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:36.694477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecec891-140c-45b9-9ed8-f7ee913057de None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.740834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 339/681] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:36.740834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77c553-6da9-40ae-ae31-b3218b4e745c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.766861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdce98-5cff-4b56-b7a2-148ad0ed8846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.768248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdce98-5cff-4b56-b7a2-148ad0ed8846 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.768736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdce98-5cff-4b56-b7a2-148ad0ed8846 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.769416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdce98-5cff-4b56-b7a2-148ad0ed8846 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.770400 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdce98-5cff-4b56-b7a2-148ad0ed8846 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:36.770919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdce98-5cff-4b56-b7a2-148ad0ed8846 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.772793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bdce98-5cff-4b56-b7a2-148ad0ed8846 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.789277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 343/682] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:36.794684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdce98-5cff-4b56-b7a2-148ad0ed8846 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.805254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:36.864944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77c553-6da9-40ae-ae31-b3218b4e745c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.867699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:36.909132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e32008-5276-4ad0-966b-0c7d0f023c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.909977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e32008-5276-4ad0-966b-0c7d0f023c79 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.910371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e32008-5276-4ad0-966b-0c7d0f023c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.910967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e32008-5276-4ad0-966b-0c7d0f023c79 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.911867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e32008-5276-4ad0-966b-0c7d0f023c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:36.912280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e32008-5276-4ad0-966b-0c7d0f023c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.913855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e32008-5276-4ad0-966b-0c7d0f023c79 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.929151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdce98-5cff-4b56-b7a2-148ad0ed8846 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:36.937631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 340/683] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:36.945659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e32008-5276-4ad0-966b-0c7d0f023c79 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:36.971887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947693e4-8710-4676-97b6-b39e53a66ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:36.972644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947693e4-8710-4676-97b6-b39e53a66ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:36.973479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947693e4-8710-4676-97b6-b39e53a66ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:36.973479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947693e4-8710-4676-97b6-b39e53a66ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:36.974293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947693e4-8710-4676-97b6-b39e53a66ca0 None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=56a94ccca15c462c9e7a5e88b907f420) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:36.974702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947693e4-8710-4676-97b6-b39e53a66ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:36.976382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947693e4-8710-4676-97b6-b39e53a66ca0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:36.987742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 344/684] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/role_assignments?user.id=56a94ccca15c462c9e7a5e88b907f420&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:37.061839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e32008-5276-4ad0-966b-0c7d0f023c79 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:37.100797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947693e4-8710-4676-97b6-b39e53a66ca0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:37.113914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628031a-e0e9-41f4-ac74-d97e09b5fe49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:37.114621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628031a-e0e9-41f4-ac74-d97e09b5fe49 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:37.114901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628031a-e0e9-41f4-ac74-d97e09b5fe49 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:37.115167 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628031a-e0e9-41f4-ac74-d97e09b5fe49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:37.125833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7628031a-e0e9-41f4-ac74-d97e09b5fe49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:37.127370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7628031a-e0e9-41f4-ac74-d97e09b5fe49 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-7628031a-e0e9-41f4-ac74-d97e09b5fe49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:37.141930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:37.143130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:37.144165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 341/685] 10.222.0.71 () {56 vars in 1093 bytes} [Wed Sep 21 12:07:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:37.151677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628031a-e0e9-41f4-ac74-d97e09b5fe49 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:37.210829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:37.252299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:37.290077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947693e4-8710-4676-97b6-b39e53a66ca0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:37.310638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628031a-e0e9-41f4-ac74-d97e09b5fe49 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:37.331120 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363c00a-90e7-4782-8960-57e76b5b84ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:37.331945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363c00a-90e7-4782-8960-57e76b5b84ec None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:37.332301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363c00a-90e7-4782-8960-57e76b5b84ec None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:37.332650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363c00a-90e7-4782-8960-57e76b5b84ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:37.345629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363c00a-90e7-4782-8960-57e76b5b84ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0363c00a-90e7-4782-8960-57e76b5b84ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:37.352442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:37.354035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:37.355144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 345/686] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:37.358477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a89cd3-4a4f-409c-92ac-682acedb432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:37.359373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a89cd3-4a4f-409c-92ac-682acedb432f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:37.359968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a89cd3-4a4f-409c-92ac-682acedb432f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:37.360252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a89cd3-4a4f-409c-92ac-682acedb432f None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:37.361000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a89cd3-4a4f-409c-92ac-682acedb432f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:37.361309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a89cd3-4a4f-409c-92ac-682acedb432f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:37.361574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363c00a-90e7-4782-8960-57e76b5b84ec None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:37.362768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a89cd3-4a4f-409c-92ac-682acedb432f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:37.387815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 342/687] 10.222.0.71 () {56 vars in 1133 bytes} [Wed Sep 21 12:07:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:37.390482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a89cd3-4a4f-409c-92ac-682acedb432f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:37.487996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:37.510763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a89cd3-4a4f-409c-92ac-682acedb432f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:37.514814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363c00a-90e7-4782-8960-57e76b5b84ec None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:37.554673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:37.560869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e777ec-815f-45bf-9e8d-824243a2dd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:37.561570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e777ec-815f-45bf-9e8d-824243a2dd97 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:37.561921 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e777ec-815f-45bf-9e8d-824243a2dd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:37.562083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25691bd-a62f-4205-b5a6-b2edd836ba47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:37.562335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e777ec-815f-45bf-9e8d-824243a2dd97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:37.562820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25691bd-a62f-4205-b5a6-b2edd836ba47 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:37.563169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25691bd-a62f-4205-b5a6-b2edd836ba47 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:37.563505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25691bd-a62f-4205-b5a6-b2edd836ba47 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:37.564261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25691bd-a62f-4205-b5a6-b2edd836ba47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:37.564633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25691bd-a62f-4205-b5a6-b2edd836ba47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:37.566265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25691bd-a62f-4205-b5a6-b2edd836ba47 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:37.578936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 346/688] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:37.584556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25691bd-a62f-4205-b5a6-b2edd836ba47 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:37.654491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e777ec-815f-45bf-9e8d-824243a2dd97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:37.656736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e777ec-815f-45bf-9e8d-824243a2dd97 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:37.717104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25691bd-a62f-4205-b5a6-b2edd836ba47 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:37.794334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a7206-580a-47d5-bec7-f001f67701cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:37.795047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a7206-580a-47d5-bec7-f001f67701cf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:37.795240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a7206-580a-47d5-bec7-f001f67701cf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:37.795518 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a7206-580a-47d5-bec7-f001f67701cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:37.819808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a7206-580a-47d5-bec7-f001f67701cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:37.823210 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a7206-580a-47d5-bec7-f001f67701cf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:37.848121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4d7a7206-580a-47d5-bec7-f001f67701cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:37.859580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:37.860787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:37.861878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:37.861878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:37.861878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:37.866411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 347/689] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:37.867684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a7206-580a-47d5-bec7-f001f67701cf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:37.894552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 343/690] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 12:07:37.900675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e777ec-815f-45bf-9e8d-824243a2dd97 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:37.965724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:38.007914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a7206-580a-47d5-bec7-f001f67701cf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:38.045380 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27eab8-125b-4512-ae95-dc1ba9706701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:38.046353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27eab8-125b-4512-ae95-dc1ba9706701 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:38.046852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27eab8-125b-4512-ae95-dc1ba9706701 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:38.047320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27eab8-125b-4512-ae95-dc1ba9706701 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:38.048201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27eab8-125b-4512-ae95-dc1ba9706701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:38.048717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27eab8-125b-4512-ae95-dc1ba9706701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:38.050287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27eab8-125b-4512-ae95-dc1ba9706701 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:38.064829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 348/691] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:38.070424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27eab8-125b-4512-ae95-dc1ba9706701 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:38.145931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e777ec-815f-45bf-9e8d-824243a2dd97 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:38.177086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27eab8-125b-4512-ae95-dc1ba9706701 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:38.179848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:38.215643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d54a44-206c-4396-8c25-182f468aecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:38.216939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d54a44-206c-4396-8c25-182f468aecfc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:38.217224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc22e5e-1ae5-4a43-9665-87d0b89edac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:38.217659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d54a44-206c-4396-8c25-182f468aecfc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:38.217863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22e5e-1ae5-4a43-9665-87d0b89edac6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:38.218149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22e5e-1ae5-4a43-9665-87d0b89edac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:38.218418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22e5e-1ae5-4a43-9665-87d0b89edac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:38.218713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d54a44-206c-4396-8c25-182f468aecfc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:38.226576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d54a44-206c-4396-8c25-182f468aecfc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:38.228981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d54a44-206c-4396-8c25-182f468aecfc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:38.236400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc22e5e-1ae5-4a43-9665-87d0b89edac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:38.237857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc22e5e-1ae5-4a43-9665-87d0b89edac6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-20d54a44-206c-4396-8c25-182f468aecfc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:38.242064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:38.242983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:38.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:38.246971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 349/692] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:38.252585 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d54a44-206c-4396-8c25-182f468aecfc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1fc22e5e-1ae5-4a43-9665-87d0b89edac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:38.264643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:38.265681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:38.266567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:38.266567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:38.266567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:38.266567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 344/693] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:38.271423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc22e5e-1ae5-4a43-9665-87d0b89edac6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:38.348197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:38.366175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d54a44-206c-4396-8c25-182f468aecfc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:38.410282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e38fd-5ad2-4159-81d7-f4a8c8413119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:38.412157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e38fd-5ad2-4159-81d7-f4a8c8413119 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:38.412671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e38fd-5ad2-4159-81d7-f4a8c8413119 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:38.414059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e38fd-5ad2-4159-81d7-f4a8c8413119 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:38.414991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e38fd-5ad2-4159-81d7-f4a8c8413119 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:38.416171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e38fd-5ad2-4159-81d7-f4a8c8413119 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:38.417790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e38fd-5ad2-4159-81d7-f4a8c8413119 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:38.446215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 350/694] 10.222.0.71 () {56 vars in 1135 bytes} [Wed Sep 21 12:07:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:38.453117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e38fd-5ad2-4159-81d7-f4a8c8413119 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:38.477027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc22e5e-1ae5-4a43-9665-87d0b89edac6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:38.481743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:38.541726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3197c-da5c-40f5-b5cc-036f2700e753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:38.542363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3197c-da5c-40f5-b5cc-036f2700e753 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:38.542628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3197c-da5c-40f5-b5cc-036f2700e753 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:38.542893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3197c-da5c-40f5-b5cc-036f2700e753 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:38.543545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3197c-da5c-40f5-b5cc-036f2700e753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:38.543878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3197c-da5c-40f5-b5cc-036f2700e753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:38.545247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3197c-da5c-40f5-b5cc-036f2700e753 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:38.565224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e38fd-5ad2-4159-81d7-f4a8c8413119 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:38.571326 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 345/695] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:38.572599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3197c-da5c-40f5-b5cc-036f2700e753 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:38.602964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5534a81-909a-4490-ba7c-b3bc0288c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:38.604571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5534a81-909a-4490-ba7c-b3bc0288c075 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:38.605063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5534a81-909a-4490-ba7c-b3bc0288c075 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:38.605525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5534a81-909a-4490-ba7c-b3bc0288c075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:38.677026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5534a81-909a-4490-ba7c-b3bc0288c075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:38.678826 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5534a81-909a-4490-ba7c-b3bc0288c075 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:38.703973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:38.796396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 351/696] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 12:07:38.803172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5534a81-909a-4490-ba7c-b3bc0288c075 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:38.822597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:38.856315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3197c-da5c-40f5-b5cc-036f2700e753 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:38.922374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5534a81-909a-4490-ba7c-b3bc0288c075 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:38.956581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a926e8-defd-4e70-ad8e-3810bd02cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:38.957241 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a926e8-defd-4e70-ad8e-3810bd02cd4d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:38.957486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a926e8-defd-4e70-ad8e-3810bd02cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:38.957766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a926e8-defd-4e70-ad8e-3810bd02cd4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:38.961660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b4131-615e-4217-8fc3-5287ba42c0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:38.962725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b4131-615e-4217-8fc3-5287ba42c0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:38.963426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:38.964110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b4131-615e-4217-8fc3-5287ba42c0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:38.964900 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b4131-615e-4217-8fc3-5287ba42c0b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:38.979455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b4131-615e-4217-8fc3-5287ba42c0b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:38.981041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a926e8-defd-4e70-ad8e-3810bd02cd4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:38.981156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b4131-615e-4217-8fc3-5287ba42c0b5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:38.982898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a926e8-defd-4e70-ad8e-3810bd02cd4d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bf9b4131-615e-4217-8fc3-5287ba42c0b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:38.997718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:38.998747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:38.999631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:38.999631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:38.999631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:38.999631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 352/697] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:39.004703 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ec9a6-9fe4-4258-864b-dca92dcb4d46 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.005077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ec9a6-9fe4-4258-864b-dca92dcb4d46 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.005409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ec9a6-9fe4-4258-864b-dca92dcb4d46 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.011102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 353/698] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:39.011102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ec9a6-9fe4-4258-864b-dca92dcb4d46 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-37a926e8-defd-4e70-ad8e-3810bd02cd4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:39.012910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:39.013957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:39.015039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:39.015039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:39.015039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:39.015039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:39.015039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:39.015039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:39.015039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:39.015039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:39.018531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 346/699] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 446 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:39.020050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60f64-0138-428d-b5ee-42e2ec23a088 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.020498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60f64-0138-428d-b5ee-42e2ec23a088 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.020938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce60f64-0138-428d-b5ee-42e2ec23a088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.048475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8ce60f64-0138-428d-b5ee-42e2ec23a088 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:39.050344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8ce60f64-0138-428d-b5ee-42e2ec23a088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:39.128381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ec9a6-9fe4-4258-864b-dca92dcb4d46 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:39.152422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 347/700] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:39.153973 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce60f64-0138-428d-b5ee-42e2ec23a088 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:39.180812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3256f-5928-426e-b256-e40bcac5e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:39.181615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3256f-5928-426e-b256-e40bcac5e68d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.181899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3256f-5928-426e-b256-e40bcac5e68d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.182191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3256f-5928-426e-b256-e40bcac5e68d None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.182893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3256f-5928-426e-b256-e40bcac5e68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:39.183193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3256f-5928-426e-b256-e40bcac5e68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:39.184592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3256f-5928-426e-b256-e40bcac5e68d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:39.204586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 354/701] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:39.210132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3256f-5928-426e-b256-e40bcac5e68d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:39.257698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:39.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:39.359388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3256f-5928-426e-b256-e40bcac5e68d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:39.380468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce60f64-0138-428d-b5ee-42e2ec23a088 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:39.407964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c40ce-7f4c-4cac-8ccb-dfd33c42c89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:39.408606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c40ce-7f4c-4cac-8ccb-dfd33c42c89e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.408879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c40ce-7f4c-4cac-8ccb-dfd33c42c89e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.409155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c40ce-7f4c-4cac-8ccb-dfd33c42c89e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.422654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c40ce-7f4c-4cac-8ccb-dfd33c42c89e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:39.424142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c40ce-7f4c-4cac-8ccb-dfd33c42c89e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-047c40ce-7f4c-4cac-8ccb-dfd33c42c89e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:39.439386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:39.440356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:39.441271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:39.441271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:39.441271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:39.441271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:39.441271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:39.441271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:39.441271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:39.441271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:39.444896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 355/702] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:39.446623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c40ce-7f4c-4cac-8ccb-dfd33c42c89e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:39.461116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474617b-1e66-46c9-b03b-5d80d94f1459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:39.461718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474617b-1e66-46c9-b03b-5d80d94f1459 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.462078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474617b-1e66-46c9-b03b-5d80d94f1459 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.462375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474617b-1e66-46c9-b03b-5d80d94f1459 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.463652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474617b-1e66-46c9-b03b-5d80d94f1459 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:39.463974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474617b-1e66-46c9-b03b-5d80d94f1459 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:39.465482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474617b-1e66-46c9-b03b-5d80d94f1459 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:39.491236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 348/703] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:39.497261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474617b-1e66-46c9-b03b-5d80d94f1459 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:39.552603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:39.560601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c40ce-7f4c-4cac-8ccb-dfd33c42c89e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:39.595822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c2c24-5aba-49a0-b251-2f93f2f47c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:39.596430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c2c24-5aba-49a0-b251-2f93f2f47c4f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.596994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c2c24-5aba-49a0-b251-2f93f2f47c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.597268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c2c24-5aba-49a0-b251-2f93f2f47c4f None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.597962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c2c24-5aba-49a0-b251-2f93f2f47c4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:39.598267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c2c24-5aba-49a0-b251-2f93f2f47c4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:39.599699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c2c24-5aba-49a0-b251-2f93f2f47c4f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:39.622365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 356/704] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:39.622365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e7f04-2075-4500-b02e-954ab71283ea None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.622365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e7f04-2075-4500-b02e-954ab71283ea None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.622365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e7f04-2075-4500-b02e-954ab71283ea None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.622365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 357/705] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:39.622365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e7f04-2075-4500-b02e-954ab71283ea None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:39.622883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474617b-1e66-46c9-b03b-5d80d94f1459 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:39.657842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641df97e-9f88-408e-b1f3-112434339872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:39.658448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641df97e-9f88-408e-b1f3-112434339872 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.658730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641df97e-9f88-408e-b1f3-112434339872 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.658999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641df97e-9f88-408e-b1f3-112434339872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.660099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641df97e-9f88-408e-b1f3-112434339872 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=463487449c1d4aada1b5f279c959b0e6) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:39.660099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641df97e-9f88-408e-b1f3-112434339872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:39.661482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641df97e-9f88-408e-b1f3-112434339872 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:39.668604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 349/706] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:39 2022] GET /identity/v3/role_assignments?user.id=463487449c1d4aada1b5f279c959b0e6&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:39.669476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641df97e-9f88-408e-b1f3-112434339872 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:39.678939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:39.763729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e7f04-2075-4500-b02e-954ab71283ea None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:39.808775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb078b6-2cd6-442d-9885-7041097c9033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:39.810256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb078b6-2cd6-442d-9885-7041097c9033 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.810718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb078b6-2cd6-442d-9885-7041097c9033 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.811142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb078b6-2cd6-442d-9885-7041097c9033 None admin] PATH_INFO: `/v3/services/network` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.811937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb078b6-2cd6-442d-9885-7041097c9033 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:39.813506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb078b6-2cd6-442d-9885-7041097c9033 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:39.823932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641df97e-9f88-408e-b1f3-112434339872 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3cb078b6-2cd6-442d-9885-7041097c9033 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:39.829824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:39.831089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 12:07:39.832125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:39.834723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 358/707] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:39 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:39.836280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc40fb0-bb1f-4bb2-8387-5f19280a491b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.836710 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc40fb0-bb1f-4bb2-8387-5f19280a491b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.837118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc40fb0-bb1f-4bb2-8387-5f19280a491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.862969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2dc40fb0-bb1f-4bb2-8387-5f19280a491b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:39.864727 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2dc40fb0-bb1f-4bb2-8387-5f19280a491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:39.881905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d66a8-9be2-4a52-ae9c-527e3c97d84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:39.882655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d66a8-9be2-4a52-ae9c-527e3c97d84f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.883155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:39.883297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d66a8-9be2-4a52-ae9c-527e3c97d84f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.883603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d66a8-9be2-4a52-ae9c-527e3c97d84f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.885883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d66a8-9be2-4a52-ae9c-527e3c97d84f None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=e0bf75c730c946bc9168dd418e7af637) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:39.886829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d66a8-9be2-4a52-ae9c-527e3c97d84f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:39.888550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20d66a8-9be2-4a52-ae9c-527e3c97d84f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:39.896920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 350/708] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:39 2022] GET /identity/v3/role_assignments?user.id=e0bf75c730c946bc9168dd418e7af637&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:39.898071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d66a8-9be2-4a52-ae9c-527e3c97d84f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:39.943775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 359/709] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:39.945220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e2a85-79d6-43b1-b222-7ac7085bf8af None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.945579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e2a85-79d6-43b1-b222-7ac7085bf8af None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.945923 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e2a85-79d6-43b1-b222-7ac7085bf8af None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.947629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 360/710] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:39.966103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db37c6d-e570-446c-92b0-6f30f99ef860 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:39.966767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db37c6d-e570-446c-92b0-6f30f99ef860 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:39.967108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db37c6d-e570-446c-92b0-6f30f99ef860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:39.999074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9db37c6d-e570-446c-92b0-6f30f99ef860 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:40.002208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9db37c6d-e570-446c-92b0-6f30f99ef860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:40.029548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:40.069315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20d66a8-9be2-4a52-ae9c-527e3c97d84f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:40.119131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10163f8-5ec3-4573-b641-7da11f5a7bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:40.119749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10163f8-5ec3-4573-b641-7da11f5a7bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:40.120059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10163f8-5ec3-4573-b641-7da11f5a7bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:40.120332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10163f8-5ec3-4573-b641-7da11f5a7bb2 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:40.120966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10163f8-5ec3-4573-b641-7da11f5a7bb2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:40.121269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10163f8-5ec3-4573-b641-7da11f5a7bb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:40.122643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10163f8-5ec3-4573-b641-7da11f5a7bb2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:40.125052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 361/711] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:07:40.128066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 351/712] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:07:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:40.136469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db37c6d-e570-446c-92b0-6f30f99ef860 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:40.253295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:40.282704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db37c6d-e570-446c-92b0-6f30f99ef860 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:40.296841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10163f8-5ec3-4573-b641-7da11f5a7bb2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:40.349899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad114472-5073-4d60-b248-dbe7a3b3bf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:40.350510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad114472-5073-4d60-b248-dbe7a3b3bf28 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:40.350786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad114472-5073-4d60-b248-dbe7a3b3bf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:40.351054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad114472-5073-4d60-b248-dbe7a3b3bf28 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:40.351683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad114472-5073-4d60-b248-dbe7a3b3bf28 None admin] RBAC: Adding query filter params (type=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:40.352004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad114472-5073-4d60-b248-dbe7a3b3bf28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:40.353438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad114472-5073-4d60-b248-dbe7a3b3bf28 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:40.368233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 362/713] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:07:40 2022] GET /identity/v3/services?type=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:40.484023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad114472-5073-4d60-b248-dbe7a3b3bf28 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:40.510811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10163f8-5ec3-4573-b641-7da11f5a7bb2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:40.564193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28508d34-9793-43b5-9106-4975817e185e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:40.564859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28508d34-9793-43b5-9106-4975817e185e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:40.565146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28508d34-9793-43b5-9106-4975817e185e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:40.565410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28508d34-9793-43b5-9106-4975817e185e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:40.566008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28508d34-9793-43b5-9106-4975817e185e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:40.567399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28508d34-9793-43b5-9106-4975817e185e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:40.576768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-28508d34-9793-43b5-9106-4975817e185e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:40.600178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:40.601494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 12:07:40.602416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:40.605434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 352/714] 10.222.0.71 () {56 vars in 1129 bytes} [Wed Sep 21 12:07:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:40.614284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28508d34-9793-43b5-9106-4975817e185e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:40.752756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28508d34-9793-43b5-9106-4975817e185e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:40.767784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad114472-5073-4d60-b248-dbe7a3b3bf28 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:40.833145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7350141-06d3-4583-8e4b-16803e0d8c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:40.833677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7350141-06d3-4583-8e4b-16803e0d8c50 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:40.833954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7350141-06d3-4583-8e4b-16803e0d8c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:40.834226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7350141-06d3-4583-8e4b-16803e0d8c50 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:40.834860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7350141-06d3-4583-8e4b-16803e0d8c50 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:40.835161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7350141-06d3-4583-8e4b-16803e0d8c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:40.836580 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7350141-06d3-4583-8e4b-16803e0d8c50 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:40.844337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d858a-845e-43da-93ca-1cee621a0ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:40.844973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d858a-845e-43da-93ca-1cee621a0ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:40.845254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d858a-845e-43da-93ca-1cee621a0ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:40.845527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d858a-845e-43da-93ca-1cee621a0ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:40.850103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 353/715] 10.222.0.71 () {56 vars in 1124 bytes} [Wed Sep 21 12:07:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:40.859208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7350141-06d3-4583-8e4b-16803e0d8c50 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:40.873367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d858a-845e-43da-93ca-1cee621a0ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:40.875084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d858a-845e-43da-93ca-1cee621a0ff0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:40.880062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c84d858a-845e-43da-93ca-1cee621a0ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:40.907415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:40.908634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:40.909511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:40.909511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:40.909511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:40.909511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 363/716] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:40.920844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d858a-845e-43da-93ca-1cee621a0ff0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:40.978459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:41.054029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7350141-06d3-4583-8e4b-16803e0d8c50 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:41.122050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211eae-fead-40a8-a7c9-e736d696a5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:41.123091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211eae-fead-40a8-a7c9-e736d696a5ab None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:41.123377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211eae-fead-40a8-a7c9-e736d696a5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:41.123649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211eae-fead-40a8-a7c9-e736d696a5ab None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:41.124523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211eae-fead-40a8-a7c9-e736d696a5ab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:41.124833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211eae-fead-40a8-a7c9-e736d696a5ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:41.126880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211eae-fead-40a8-a7c9-e736d696a5ab None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:41.131560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d858a-845e-43da-93ca-1cee621a0ff0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:41.144517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 354/717] 10.222.0.71 () {56 vars in 1124 bytes} [Wed Sep 21 12:07:40 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:41.196903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd94ec-b1dd-4362-88bb-bd61213c7929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:41.197807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd94ec-b1dd-4362-88bb-bd61213c7929 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:41.198260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd94ec-b1dd-4362-88bb-bd61213c7929 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:41.198679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd94ec-b1dd-4362-88bb-bd61213c7929 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:41.199489 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd94ec-b1dd-4362-88bb-bd61213c7929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:41.199965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd94ec-b1dd-4362-88bb-bd61213c7929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:41.201513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd94ec-b1dd-4362-88bb-bd61213c7929 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:41.223466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 364/718] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:41.229572 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211eae-fead-40a8-a7c9-e736d696a5ab None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:41.380254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211eae-fead-40a8-a7c9-e736d696a5ab None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:41.416986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:41.429507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8629-602e-4de9-9400-1ff0ecda7e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:41.429507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8629-602e-4de9-9400-1ff0ecda7e5e None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:41.429507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8629-602e-4de9-9400-1ff0ecda7e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:41.429507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8629-602e-4de9-9400-1ff0ecda7e5e None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:41.429940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8629-602e-4de9-9400-1ff0ecda7e5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:41.431250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a8629-602e-4de9-9400-1ff0ecda7e5e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:41.460790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 355/719] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:07:41 2022] POST /identity/v3/users => generated 254 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:42.468005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50540f19-1ca0-4ce2-a79d-d8bda0c0613b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:42.468530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50540f19-1ca0-4ce2-a79d-d8bda0c0613b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:42.468530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50540f19-1ca0-4ce2-a79d-d8bda0c0613b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:42.470153 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 365/720] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:42.475286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8d3bf-5607-492d-84dd-d33e437e96c6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:42.475695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8d3bf-5607-492d-84dd-d33e437e96c6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:42.475942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8d3bf-5607-492d-84dd-d33e437e96c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:42.499444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-51c8d3bf-5607-492d-84dd-d33e437e96c6 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:42.500663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-51c8d3bf-5607-492d-84dd-d33e437e96c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:42.585378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 356/721] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:07:42.842303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4089c4f-00be-471c-a626-2a5b4ad976cb None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:42.842582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4089c4f-00be-471c-a626-2a5b4ad976cb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:42.842855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4089c4f-00be-471c-a626-2a5b4ad976cb None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:42.844542 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 366/722] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:42.857877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6981dfa-2108-4fb7-adda-de8a13472f98 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:42.858228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6981dfa-2108-4fb7-adda-de8a13472f98 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:42.858435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6981dfa-2108-4fb7-adda-de8a13472f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:42.884768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b6981dfa-2108-4fb7-adda-de8a13472f98 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:42.885975 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b6981dfa-2108-4fb7-adda-de8a13472f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:42.973468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 357/723] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:42.983714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4089c4f-00be-471c-a626-2a5b4ad976cb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:43.002710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bd8e1-c77c-42bd-a286-c67ccf796560 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:43.003045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bd8e1-c77c-42bd-a286-c67ccf796560 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:43.003255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bd8e1-c77c-42bd-a286-c67ccf796560 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:43.004735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 358/724] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:43.010438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec7c4a-14c8-4fd8-837f-c1aff0e1c1c4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:43.011093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec7c4a-14c8-4fd8-837f-c1aff0e1c1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:43.011371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec7c4a-14c8-4fd8-837f-c1aff0e1c1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:43.043254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f9ec7c4a-14c8-4fd8-837f-c1aff0e1c1c4 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:43.044403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f9ec7c4a-14c8-4fd8-837f-c1aff0e1c1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:43.092149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:43.116346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:43.147607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 359/725] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 12:07:43.184709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4089c4f-00be-471c-a626-2a5b4ad976cb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:43.236917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f4faa-9d31-4e52-883b-8eb7af8287e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:43.237545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f4faa-9d31-4e52-883b-8eb7af8287e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:43.237825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f4faa-9d31-4e52-883b-8eb7af8287e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:43.238160 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f4faa-9d31-4e52-883b-8eb7af8287e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:43.252602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f4faa-9d31-4e52-883b-8eb7af8287e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c06f4faa-9d31-4e52-883b-8eb7af8287e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:43.257344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:43.258447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:43.260019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:43.261703 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 367/726] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:43.268835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec7c4a-14c8-4fd8-837f-c1aff0e1c1c4 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:43.365449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f4faa-9d31-4e52-883b-8eb7af8287e2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:43.393969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec7c4a-14c8-4fd8-837f-c1aff0e1c1c4 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:43.467338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc536189-6dd3-478b-90ba-b07e63e0abf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:43.467981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc536189-6dd3-478b-90ba-b07e63e0abf4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:43.468255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc536189-6dd3-478b-90ba-b07e63e0abf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:43.468522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc536189-6dd3-478b-90ba-b07e63e0abf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:43.469197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc536189-6dd3-478b-90ba-b07e63e0abf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:43.472415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc536189-6dd3-478b-90ba-b07e63e0abf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:43.473818 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc536189-6dd3-478b-90ba-b07e63e0abf4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:43.497846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 360/727] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:43.508780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc536189-6dd3-478b-90ba-b07e63e0abf4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:43.545076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:43.546214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f4faa-9d31-4e52-883b-8eb7af8287e2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:43.564650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:43.592247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ba97b-f0d0-45d3-80eb-5982211d0d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:43.593934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ba97b-f0d0-45d3-80eb-5982211d0d09 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:43.594671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ba97b-f0d0-45d3-80eb-5982211d0d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:43.595403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ba97b-f0d0-45d3-80eb-5982211d0d09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:43.611034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ba97b-f0d0-45d3-80eb-5982211d0d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ff0ba97b-f0d0-45d3-80eb-5982211d0d09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:43.618749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:43.619740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:43.620690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:43.623206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 368/728] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:43.624203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ba97b-f0d0-45d3-80eb-5982211d0d09 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:43.626075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc536189-6dd3-478b-90ba-b07e63e0abf4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:43.666662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca865f-6418-497a-b3fd-77d7b6ec0c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:43.667551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca865f-6418-497a-b3fd-77d7b6ec0c6d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:43.667958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca865f-6418-497a-b3fd-77d7b6ec0c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:43.668355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca865f-6418-497a-b3fd-77d7b6ec0c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:43.683059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca865f-6418-497a-b3fd-77d7b6ec0c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:43.684930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ca865f-6418-497a-b3fd-77d7b6ec0c6d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-09ca865f-6418-497a-b3fd-77d7b6ec0c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:43.700443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:43.701471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:43.702369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:43.702369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:43.702369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:43.702369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 361/729] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:43.707420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca865f-6418-497a-b3fd-77d7b6ec0c6d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:43.774118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ba97b-f0d0-45d3-80eb-5982211d0d09 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:43.817859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:43.820062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0121f-9d76-4b34-83f0-f5be7165eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:43.821124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0121f-9d76-4b34-83f0-f5be7165eaf2 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:43.821615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0121f-9d76-4b34-83f0-f5be7165eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:43.822517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0121f-9d76-4b34-83f0-f5be7165eaf2 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:43.824181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0121f-9d76-4b34-83f0-f5be7165eaf2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:43.826268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0121f-9d76-4b34-83f0-f5be7165eaf2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:43.835538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca865f-6418-497a-b3fd-77d7b6ec0c6d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:43.851433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 369/730] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:07:43 2022] POST /identity/v3/services => generated 240 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:43.852632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0121f-9d76-4b34-83f0-f5be7165eaf2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:43.900156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:43.916764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6694d-6df8-48e4-9a4e-bc92c0d53da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:43.917561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6694d-6df8-48e4-9a4e-bc92c0d53da3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:43.917927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6694d-6df8-48e4-9a4e-bc92c0d53da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:43.918200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6694d-6df8-48e4-9a4e-bc92c0d53da3 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:43.918981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6694d-6df8-48e4-9a4e-bc92c0d53da3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:43.919361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6694d-6df8-48e4-9a4e-bc92c0d53da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:43.920892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6694d-6df8-48e4-9a4e-bc92c0d53da3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:43.936075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 362/731] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:43.937625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06000b-847f-4049-affb-005868fa362d None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:43.937995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06000b-847f-4049-affb-005868fa362d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:43.938351 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06000b-847f-4049-affb-005868fa362d None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:43.940141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 363/732] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:43.941027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06000b-847f-4049-affb-005868fa362d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:43.981049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0121f-9d76-4b34-83f0-f5be7165eaf2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:44.019864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623bfd6-cc50-4469-9a87-69ecb5bf3606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:44.020873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623bfd6-cc50-4469-9a87-69ecb5bf3606 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.021326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623bfd6-cc50-4469-9a87-69ecb5bf3606 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.021805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623bfd6-cc50-4469-9a87-69ecb5bf3606 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.022608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623bfd6-cc50-4469-9a87-69ecb5bf3606 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:44.023061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623bfd6-cc50-4469-9a87-69ecb5bf3606 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:44.024690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623bfd6-cc50-4469-9a87-69ecb5bf3606 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:44.043085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 370/733] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:44.044769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cefe2e-97c4-4450-9443-7b948dc16167 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.045204 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cefe2e-97c4-4450-9443-7b948dc16167 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.045762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cefe2e-97c4-4450-9443-7b948dc16167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.060140 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06000b-847f-4049-affb-005868fa362d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:44.065535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c0cefe2e-97c4-4450-9443-7b948dc16167 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:44.066730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c0cefe2e-97c4-4450-9443-7b948dc16167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:44.099316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236deea-726f-4f4b-a126-8fd5deee526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:44.100077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236deea-726f-4f4b-a126-8fd5deee526f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.100424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236deea-726f-4f4b-a126-8fd5deee526f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.100845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236deea-726f-4f4b-a126-8fd5deee526f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.114587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236deea-726f-4f4b-a126-8fd5deee526f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:44.116084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236deea-726f-4f4b-a126-8fd5deee526f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:44.119458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6236deea-726f-4f4b-a126-8fd5deee526f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:44.133843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:44.135525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:44.136926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:44.136926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:44.136926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:44.136926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 364/734] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:44.139055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236deea-726f-4f4b-a126-8fd5deee526f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:44.156229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 371/735] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:44 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:44.157582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cefe2e-97c4-4450-9443-7b948dc16167 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:44.191046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:44.279386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cefe2e-97c4-4450-9443-7b948dc16167 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:44.311586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236deea-726f-4f4b-a126-8fd5deee526f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:44.338621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276d42-7926-4350-b399-d7bf6bc39000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:44.339243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276d42-7926-4350-b399-d7bf6bc39000 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.339562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276d42-7926-4350-b399-d7bf6bc39000 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.339864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51276d42-7926-4350-b399-d7bf6bc39000 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.340520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276d42-7926-4350-b399-d7bf6bc39000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:44.340822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276d42-7926-4350-b399-d7bf6bc39000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:44.342468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51276d42-7926-4350-b399-d7bf6bc39000 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:44.361790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 372/736] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:44.368085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276d42-7926-4350-b399-d7bf6bc39000 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:44.380530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde4c5f-8d1a-4c0e-84f2-29f5252a35b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:44.382326 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde4c5f-8d1a-4c0e-84f2-29f5252a35b1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.382610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde4c5f-8d1a-4c0e-84f2-29f5252a35b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.382884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde4c5f-8d1a-4c0e-84f2-29f5252a35b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.397107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde4c5f-8d1a-4c0e-84f2-29f5252a35b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:44.398802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde4c5f-8d1a-4c0e-84f2-29f5252a35b1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-abde4c5f-8d1a-4c0e-84f2-29f5252a35b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:44.418603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:44.419659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:44.420852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:44.423986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 365/737] 10.222.0.71 () {56 vars in 1093 bytes} [Wed Sep 21 12:07:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:44.425218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839e7de-bebe-40ad-9102-0bd4af31dc8a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.425500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839e7de-bebe-40ad-9102-0bd4af31dc8a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.425774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839e7de-bebe-40ad-9102-0bd4af31dc8a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.427267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 366/738] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:44.428499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839e7de-bebe-40ad-9102-0bd4af31dc8a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:44.493255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:44.500124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:44.519892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51276d42-7926-4350-b399-d7bf6bc39000 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:44.542914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839e7de-bebe-40ad-9102-0bd4af31dc8a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:44.556336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b63d2-266d-49f2-89e6-b78584b257a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:44.557085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b63d2-266d-49f2-89e6-b78584b257a0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.557280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b63d2-266d-49f2-89e6-b78584b257a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.557518 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b63d2-266d-49f2-89e6-b78584b257a0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.566384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b63d2-266d-49f2-89e6-b78584b257a0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:44.567857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b63d2-266d-49f2-89e6-b78584b257a0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:44.578777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1794bd6-4da8-49ca-b7af-d2986e515ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:44.579383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1794bd6-4da8-49ca-b7af-d2986e515ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9a4b63d2-266d-49f2-89e6-b78584b257a0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:44.579619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:44.580716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:44.586694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:44.588150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1794bd6-4da8-49ca-b7af-d2986e515ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.588150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1794bd6-4da8-49ca-b7af-d2986e515ea2 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.588150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1794bd6-4da8-49ca-b7af-d2986e515ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:44.588150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1794bd6-4da8-49ca-b7af-d2986e515ea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:44.588414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 373/739] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:44.588414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86828e84-7c91-48ff-8a31-99091a489941 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.588414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86828e84-7c91-48ff-8a31-99091a489941 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.588806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1794bd6-4da8-49ca-b7af-d2986e515ea2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:44.588878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86828e84-7c91-48ff-8a31-99091a489941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.602408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-86828e84-7c91-48ff-8a31-99091a489941 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:44.604294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86828e84-7c91-48ff-8a31-99091a489941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:44.610161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 367/740] 10.222.0.71 () {56 vars in 1133 bytes} [Wed Sep 21 12:07:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:44.610161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1794bd6-4da8-49ca-b7af-d2986e515ea2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:44.679889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 374/741] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:44 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 12:07:44.679889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86828e84-7c91-48ff-8a31-99091a489941 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:44.731559 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1794bd6-4da8-49ca-b7af-d2986e515ea2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:44.756368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:44.774550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:44.785698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2f036-394e-4fe7-8601-e838bfeee45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:44.786719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f036-394e-4fe7-8601-e838bfeee45b None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.787108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f036-394e-4fe7-8601-e838bfeee45b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.787488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f036-394e-4fe7-8601-e838bfeee45b None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.788682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2f036-394e-4fe7-8601-e838bfeee45b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:44.790155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2f036-394e-4fe7-8601-e838bfeee45b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:44.806954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86828e84-7c91-48ff-8a31-99091a489941 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:44.812107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 368/742] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:07:44 2022] POST /identity/v3/services => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:44.813166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2f036-394e-4fe7-8601-e838bfeee45b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:44.874297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41077c-c687-4e2d-8781-e958add36730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:44.875344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41077c-c687-4e2d-8781-e958add36730 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.875787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41077c-c687-4e2d-8781-e958add36730 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.876203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41077c-c687-4e2d-8781-e958add36730 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:44.877132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41077c-c687-4e2d-8781-e958add36730 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:44.877586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41077c-c687-4e2d-8781-e958add36730 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:44.879309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41077c-c687-4e2d-8781-e958add36730 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:44.916168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 375/743] 10.222.0.71 () {56 vars in 1135 bytes} [Wed Sep 21 12:07:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:44.921844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41077c-c687-4e2d-8781-e958add36730 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:44.936238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2f036-394e-4fe7-8601-e838bfeee45b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:44.992343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201ba42-f7a0-4541-8cd5-a269012789cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:44.995994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201ba42-f7a0-4541-8cd5-a269012789cb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:44.996273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201ba42-f7a0-4541-8cd5-a269012789cb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:44.996856 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201ba42-f7a0-4541-8cd5-a269012789cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.011172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201ba42-f7a0-4541-8cd5-a269012789cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:45.013143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201ba42-f7a0-4541-8cd5-a269012789cb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d201ba42-f7a0-4541-8cd5-a269012789cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:45.032745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:45.034467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:45.035606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:45.035606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:45.035606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:45.037268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 369/744] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:45.038488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201ba42-f7a0-4541-8cd5-a269012789cb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:45.075848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:45.083654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41077c-c687-4e2d-8781-e958add36730 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:45.117056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:45.138769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ce143-ea81-4504-bc8d-6e2d00ea8e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:45.139398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ce143-ea81-4504-bc8d-6e2d00ea8e19 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:45.140140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ce143-ea81-4504-bc8d-6e2d00ea8e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:45.140419 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ce143-ea81-4504-bc8d-6e2d00ea8e19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.168696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201ba42-f7a0-4541-8cd5-a269012789cb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:45.219086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957566b-f256-4338-ba28-cf2bed229aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:45.219658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957566b-f256-4338-ba28-cf2bed229aea None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:45.219967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957566b-f256-4338-ba28-cf2bed229aea None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:45.220529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957566b-f256-4338-ba28-cf2bed229aea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.223986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ce143-ea81-4504-bc8d-6e2d00ea8e19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:45.225408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ce143-ea81-4504-bc8d-6e2d00ea8e19 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:45.234460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957566b-f256-4338-ba28-cf2bed229aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5957566b-f256-4338-ba28-cf2bed229aea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:45.242162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:45.243115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:45.244527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 370/745] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:45.245213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957566b-f256-4338-ba28-cf2bed229aea None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:45.341522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 376/746] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 12:07:45.342482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ce143-ea81-4504-bc8d-6e2d00ea8e19 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:45.370280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:45.375105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:45.400156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957566b-f256-4338-ba28-cf2bed229aea None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:45.451002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc3aca-6c35-4afc-ab69-fae95e7177f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:45.451616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc3aca-6c35-4afc-ab69-fae95e7177f1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:45.452121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc3aca-6c35-4afc-ab69-fae95e7177f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:45.452396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc3aca-6c35-4afc-ab69-fae95e7177f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.453040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc3aca-6c35-4afc-ab69-fae95e7177f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:45.453342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc3aca-6c35-4afc-ab69-fae95e7177f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:45.454716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc3aca-6c35-4afc-ab69-fae95e7177f1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:45.466895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ce143-ea81-4504-bc8d-6e2d00ea8e19 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:45.468294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 371/747] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:45.469299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc3aca-6c35-4afc-ab69-fae95e7177f1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:45.510423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51380b2c-5d42-4078-9318-28861c5edc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:45.511493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51380b2c-5d42-4078-9318-28861c5edc10 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:45.512110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51380b2c-5d42-4078-9318-28861c5edc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:45.512591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51380b2c-5d42-4078-9318-28861c5edc10 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.513511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51380b2c-5d42-4078-9318-28861c5edc10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:45.514087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51380b2c-5d42-4078-9318-28861c5edc10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:45.515818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51380b2c-5d42-4078-9318-28861c5edc10 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:45.529863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 377/748] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:45.531228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51380b2c-5d42-4078-9318-28861c5edc10 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:45.573937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc3aca-6c35-4afc-ab69-fae95e7177f1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:45.609660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5da18-13e8-4969-b69c-eda5f665a5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:45.610300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5da18-13e8-4969-b69c-eda5f665a5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:45.610583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5da18-13e8-4969-b69c-eda5f665a5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:45.610856 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a5da18-13e8-4969-b69c-eda5f665a5e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.622154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5da18-13e8-4969-b69c-eda5f665a5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:45.623805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a5da18-13e8-4969-b69c-eda5f665a5e6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:45.631212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:45.638931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c3a5da18-13e8-4969-b69c-eda5f665a5e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:45.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:45.642780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:45.643661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:45.643661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:45.643661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:45.648446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 372/749] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:45.649363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5da18-13e8-4969-b69c-eda5f665a5e6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:45.656051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51380b2c-5d42-4078-9318-28861c5edc10 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:45.695597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd009a9-2cf9-46fe-92e5-7277e1cd81ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:45.696773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd009a9-2cf9-46fe-92e5-7277e1cd81ee None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:45.697281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd009a9-2cf9-46fe-92e5-7277e1cd81ee None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:45.697755 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd009a9-2cf9-46fe-92e5-7277e1cd81ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.705772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd009a9-2cf9-46fe-92e5-7277e1cd81ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:45.707609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd009a9-2cf9-46fe-92e5-7277e1cd81ee None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6fd009a9-2cf9-46fe-92e5-7277e1cd81ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:45.721122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:45.722031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:45.722908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:45.722908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:45.722908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:45.722908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:45.722908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:45.722908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:45.722908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:45.722908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:45.726479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 378/750] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:45.727688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd009a9-2cf9-46fe-92e5-7277e1cd81ee None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:45.765281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a5da18-13e8-4969-b69c-eda5f665a5e6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:45.801622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031ac4b-8845-47f6-8d1d-75eeec9883b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:45.802264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ac4b-8845-47f6-8d1d-75eeec9883b2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:45.802545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ac4b-8845-47f6-8d1d-75eeec9883b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:45.802820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031ac4b-8845-47f6-8d1d-75eeec9883b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.814598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031ac4b-8845-47f6-8d1d-75eeec9883b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:45.816049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031ac4b-8845-47f6-8d1d-75eeec9883b2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0031ac4b-8845-47f6-8d1d-75eeec9883b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:45.831427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:45.832474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:45.833369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:45.833369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:45.833369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:45.837445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 373/751] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:45.838367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031ac4b-8845-47f6-8d1d-75eeec9883b2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:45.853765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd009a9-2cf9-46fe-92e5-7277e1cd81ee None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:45.893542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:45.895625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9e618-a7fe-4648-b716-835d18628d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:45.897368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9e618-a7fe-4648-b716-835d18628d24 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:45.897895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9e618-a7fe-4648-b716-835d18628d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:45.898456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9e618-a7fe-4648-b716-835d18628d24 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.899320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9e618-a7fe-4648-b716-835d18628d24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:45.899947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9e618-a7fe-4648-b716-835d18628d24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:45.902405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9e618-a7fe-4648-b716-835d18628d24 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:45.916685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 379/752] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:45.917835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9e618-a7fe-4648-b716-835d18628d24 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:45.931780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:45.949361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031ac4b-8845-47f6-8d1d-75eeec9883b2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:45.984107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9fc78-321c-4f7b-b54a-7fc31154eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:45.984641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9fc78-321c-4f7b-b54a-7fc31154eab2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:45.984879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9fc78-321c-4f7b-b54a-7fc31154eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:45.985147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9fc78-321c-4f7b-b54a-7fc31154eab2 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:45.985834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9fc78-321c-4f7b-b54a-7fc31154eab2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:45.986138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9fc78-321c-4f7b-b54a-7fc31154eab2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:45.987556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9fc78-321c-4f7b-b54a-7fc31154eab2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:46.000919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 374/753] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:46.001809 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9fc78-321c-4f7b-b54a-7fc31154eab2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:46.034460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9e618-a7fe-4648-b716-835d18628d24 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:46.073860 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e646da-dd16-464f-a1eb-a6e84e86459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:46.074842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e646da-dd16-464f-a1eb-a6e84e86459f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.075330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e646da-dd16-464f-a1eb-a6e84e86459f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.075868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e646da-dd16-464f-a1eb-a6e84e86459f None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.076823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e646da-dd16-464f-a1eb-a6e84e86459f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:46.077372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e646da-dd16-464f-a1eb-a6e84e86459f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:46.079093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e646da-dd16-464f-a1eb-a6e84e86459f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:46.093612 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 380/754] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:46.094925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e646da-dd16-464f-a1eb-a6e84e86459f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:46.112697 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9fc78-321c-4f7b-b54a-7fc31154eab2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:46.125460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:46.151686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463c9db-f2c2-4ab3-8b6e-affb6899f297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:46.152353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463c9db-f2c2-4ab3-8b6e-affb6899f297 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.152633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463c9db-f2c2-4ab3-8b6e-affb6899f297 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.152907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463c9db-f2c2-4ab3-8b6e-affb6899f297 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.159855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463c9db-f2c2-4ab3-8b6e-affb6899f297 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:46.162035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463c9db-f2c2-4ab3-8b6e-affb6899f297 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6463c9db-f2c2-4ab3-8b6e-affb6899f297 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:46.177093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:46.178044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 375/755] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:46.180016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463c9db-f2c2-4ab3-8b6e-affb6899f297 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:46.191406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:46.230372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e646da-dd16-464f-a1eb-a6e84e86459f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:46.266557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7849f88-335d-49d5-8f5a-c719cfc6358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:46.267134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7849f88-335d-49d5-8f5a-c719cfc6358a None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.267412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7849f88-335d-49d5-8f5a-c719cfc6358a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.268079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7849f88-335d-49d5-8f5a-c719cfc6358a None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/463487449c1d4aada1b5f279c959b0e6/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.270341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7849f88-335d-49d5-8f5a-c719cfc6358a None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=463487449c1d4aada1b5f279c959b0e6, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:46.271803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7849f88-335d-49d5-8f5a-c719cfc6358a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:46.284895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 381/756] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:07:46 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/463487449c1d4aada1b5f279c959b0e6/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:46.285915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7849f88-335d-49d5-8f5a-c719cfc6358a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:46.305354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463c9db-f2c2-4ab3-8b6e-affb6899f297 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:46.345922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced666cf-a6bc-4ef8-8896-2dc8207ecbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:46.347028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced666cf-a6bc-4ef8-8896-2dc8207ecbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.347523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced666cf-a6bc-4ef8-8896-2dc8207ecbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.348115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced666cf-a6bc-4ef8-8896-2dc8207ecbf9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.358358 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced666cf-a6bc-4ef8-8896-2dc8207ecbf9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:46.360513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced666cf-a6bc-4ef8-8896-2dc8207ecbf9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ced666cf-a6bc-4ef8-8896-2dc8207ecbf9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:46.377175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:46.378343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:07:46.380394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:46.388026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 376/757] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:07:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:46.389892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dfd5e-06f9-4efa-8212-b13ee8a8d955 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.390337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dfd5e-06f9-4efa-8212-b13ee8a8d955 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.390744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1dfd5e-06f9-4efa-8212-b13ee8a8d955 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.392642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 377/758] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:46.393708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dfd5e-06f9-4efa-8212-b13ee8a8d955 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:46.439564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:46.457344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7849f88-335d-49d5-8f5a-c719cfc6358a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:46.494205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:46.501606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e21bc-c57b-43fd-81b4-fb7d57bf9bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:46.502278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e21bc-c57b-43fd-81b4-fb7d57bf9bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.502558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e21bc-c57b-43fd-81b4-fb7d57bf9bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.502850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e21bc-c57b-43fd-81b4-fb7d57bf9bc4 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.503593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e21bc-c57b-43fd-81b4-fb7d57bf9bc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:46.504382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e21bc-c57b-43fd-81b4-fb7d57bf9bc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:46.505431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e21bc-c57b-43fd-81b4-fb7d57bf9bc4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:46.523891 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 382/759] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:46.525647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ed186-8353-4cc9-bbee-8cce0bad05c4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.525928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ed186-8353-4cc9-bbee-8cce0bad05c4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.526205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ed186-8353-4cc9-bbee-8cce0bad05c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.546691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1dfd5e-06f9-4efa-8212-b13ee8a8d955 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:46.547387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fc9ed186-8353-4cc9-bbee-8cce0bad05c4 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:46.548998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fc9ed186-8353-4cc9-bbee-8cce0bad05c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:46.587460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8358e-8d88-4a9e-8cc8-9a10488e9f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:46.588146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8358e-8d88-4a9e-8cc8-9a10488e9f5f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.588413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8358e-8d88-4a9e-8cc8-9a10488e9f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.588704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8358e-8d88-4a9e-8cc8-9a10488e9f5f None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.589417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8358e-8d88-4a9e-8cc8-9a10488e9f5f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:46.589792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8358e-8d88-4a9e-8cc8-9a10488e9f5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:46.591450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8358e-8d88-4a9e-8cc8-9a10488e9f5f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:46.612074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 378/760] 10.222.0.71 () {56 vars in 1149 bytes} [Wed Sep 21 12:07:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:46.617460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8358e-8d88-4a9e-8cc8-9a10488e9f5f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:46.628050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 383/761] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:07:46.628733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ed186-8353-4cc9-bbee-8cce0bad05c4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:46.739462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:46.755359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8358e-8d88-4a9e-8cc8-9a10488e9f5f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:46.759649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ed186-8353-4cc9-bbee-8cce0bad05c4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:46.787952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:46.816536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823535db-028b-44c4-91cf-241f2a63a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:46.817154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823535db-028b-44c4-91cf-241f2a63a702 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.817434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823535db-028b-44c4-91cf-241f2a63a702 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.817708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823535db-028b-44c4-91cf-241f2a63a702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.818441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823535db-028b-44c4-91cf-241f2a63a702 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=e0bf75c730c946bc9168dd418e7af637) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:46.821081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c914fa-c157-4920-8236-920185a203e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:46.822249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c914fa-c157-4920-8236-920185a203e7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:46.822524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c914fa-c157-4920-8236-920185a203e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:46.822790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c914fa-c157-4920-8236-920185a203e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:46.823614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823535db-028b-44c4-91cf-241f2a63a702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:46.825083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823535db-028b-44c4-91cf-241f2a63a702 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:46.843977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 379/762] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:46 2022] GET /identity/v3/role_assignments?user.id=e0bf75c730c946bc9168dd418e7af637&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:46.940640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c914fa-c157-4920-8236-920185a203e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:46.942513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c914fa-c157-4920-8236-920185a203e7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:46.990808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823535db-028b-44c4-91cf-241f2a63a702 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:47.156620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823535db-028b-44c4-91cf-241f2a63a702 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:47.169505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 384/763] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:07:47.173699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c914fa-c157-4920-8236-920185a203e7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:47.204540 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16189281-783d-4217-9665-2853f386d834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:47.205586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16189281-783d-4217-9665-2853f386d834 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:47.206129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16189281-783d-4217-9665-2853f386d834 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:47.206606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16189281-783d-4217-9665-2853f386d834 None admin] PATH_INFO: `/v3/services/network` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:47.207396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16189281-783d-4217-9665-2853f386d834 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:47.209003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16189281-783d-4217-9665-2853f386d834 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:47.218147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-16189281-783d-4217-9665-2853f386d834 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:47.232220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:47.233268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 12:07:47.234093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:47.241716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 380/764] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:46 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:47.245341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16189281-783d-4217-9665-2853f386d834 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:47.272015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:47.359023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c914fa-c157-4920-8236-920185a203e7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:47.381376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16189281-783d-4217-9665-2853f386d834 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:47.400112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5c14d-f3ba-45c6-b8aa-ef971576bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:47.401268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5c14d-f3ba-45c6-b8aa-ef971576bf46 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:47.401462 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5c14d-f3ba-45c6-b8aa-ef971576bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:47.401601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5c14d-f3ba-45c6-b8aa-ef971576bf46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:47.414057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5c14d-f3ba-45c6-b8aa-ef971576bf46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:47.416217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5c14d-f3ba-45c6-b8aa-ef971576bf46 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:47.424417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152e9e2-651b-4fa8-8437-8a00b85d6f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:47.425490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152e9e2-651b-4fa8-8437-8a00b85d6f27 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:47.425985 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152e9e2-651b-4fa8-8437-8a00b85d6f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:47.426498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152e9e2-651b-4fa8-8437-8a00b85d6f27 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:47.427371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152e9e2-651b-4fa8-8437-8a00b85d6f27 None admin] RBAC: Adding query filter params (name=network) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:47.427942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152e9e2-651b-4fa8-8437-8a00b85d6f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:47.429591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152e9e2-651b-4fa8-8437-8a00b85d6f27 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0bb5c14d-f3ba-45c6-b8aa-ef971576bf46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:47.434762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:47.435725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:47.436702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:47.436702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:47.436702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:47.436702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 385/765] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:47.437947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c71111-fb08-42ee-80c4-3df4657e8b76 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:47.438237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c71111-fb08-42ee-80c4-3df4657e8b76 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:47.438561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c71111-fb08-42ee-80c4-3df4657e8b76 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:47.439272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 381/766] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:07:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:47.439948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 386/767] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:47.441661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152e9e2-651b-4fa8-8437-8a00b85d6f27 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:47.445071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54065d37-7a1e-47a9-9388-a5d79c0fcbbd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:47.445359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54065d37-7a1e-47a9-9388-a5d79c0fcbbd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:47.445701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54065d37-7a1e-47a9-9388-a5d79c0fcbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:47.466958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-54065d37-7a1e-47a9-9388-a5d79c0fcbbd None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:47.468202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-54065d37-7a1e-47a9-9388-a5d79c0fcbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:47.530607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:47.552058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 387/768] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:07:47.552958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152e9e2-651b-4fa8-8437-8a00b85d6f27 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:47.553151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54065d37-7a1e-47a9-9388-a5d79c0fcbbd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:47.555594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:47.594212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0880b16-bc1c-42a6-bbf8-b60b69335fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:47.594212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0880b16-bc1c-42a6-bbf8-b60b69335fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:47.594212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0880b16-bc1c-42a6-bbf8-b60b69335fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:47.594212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0880b16-bc1c-42a6-bbf8-b60b69335fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:47.594918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0880b16-bc1c-42a6-bbf8-b60b69335fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:47.594918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0880b16-bc1c-42a6-bbf8-b60b69335fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:47.594918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0880b16-bc1c-42a6-bbf8-b60b69335fc5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:47.607528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 382/769] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:47.612740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0880b16-bc1c-42a6-bbf8-b60b69335fc5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:47.723110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54065d37-7a1e-47a9-9388-a5d79c0fcbbd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:47.802325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b531d77-a20c-4c44-a5d9-6c751d9f059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:47.803841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b531d77-a20c-4c44-a5d9-6c751d9f059a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:47.804341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b531d77-a20c-4c44-a5d9-6c751d9f059a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:47.804853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b531d77-a20c-4c44-a5d9-6c751d9f059a None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:47.805699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b531d77-a20c-4c44-a5d9-6c751d9f059a None admin] RBAC: Adding query filter params (type=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:47.807389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b531d77-a20c-4c44-a5d9-6c751d9f059a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:47.809745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b531d77-a20c-4c44-a5d9-6c751d9f059a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:47.818102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0880b16-bc1c-42a6-bbf8-b60b69335fc5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:47.820201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 388/770] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:07:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:47.826657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b531d77-a20c-4c44-a5d9-6c751d9f059a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:47.877466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae6857-3859-4b27-a888-24f7a4c05c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:47.878125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6857-3859-4b27-a888-24f7a4c05c66 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:47.878405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6857-3859-4b27-a888-24f7a4c05c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:47.878676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae6857-3859-4b27-a888-24f7a4c05c66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:47.893662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae6857-3859-4b27-a888-24f7a4c05c66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:47.900738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae6857-3859-4b27-a888-24f7a4c05c66 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:47.905442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-62ae6857-3859-4b27-a888-24f7a4c05c66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:47.915148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:47.916117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:47.917007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:47.917007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:47.917007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:47.917007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:47.917007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:47.917007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:47.917007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:47.917007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:47.917007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 383/771] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:47.917485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae6857-3859-4b27-a888-24f7a4c05c66 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:47.935352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:47.976925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b531d77-a20c-4c44-a5d9-6c751d9f059a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:48.027113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245db4de-e6b2-4ff5-be95-c6a53e9f0d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:48.027722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245db4de-e6b2-4ff5-be95-c6a53e9f0d2a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.028262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245db4de-e6b2-4ff5-be95-c6a53e9f0d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.029011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245db4de-e6b2-4ff5-be95-c6a53e9f0d2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.030796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245db4de-e6b2-4ff5-be95-c6a53e9f0d2a None admin] RBAC: Adding query filter params (interface=public, service_id=dccef50d0d994ed1bb6ec982cee2efce, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:48.031151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245db4de-e6b2-4ff5-be95-c6a53e9f0d2a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:48.032886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245db4de-e6b2-4ff5-be95-c6a53e9f0d2a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:48.039538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae6857-3859-4b27-a888-24f7a4c05c66 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:48.047830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 389/772] 10.222.0.71 () {56 vars in 1251 bytes} [Wed Sep 21 12:07:47 2022] GET /identity/v3/endpoints?service_id=dccef50d0d994ed1bb6ec982cee2efce&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:48.050706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245db4de-e6b2-4ff5-be95-c6a53e9f0d2a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:48.083427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c43b7c-121b-4282-a178-edb50d60e716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:48.084419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c43b7c-121b-4282-a178-edb50d60e716 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.084704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c43b7c-121b-4282-a178-edb50d60e716 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.084979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c43b7c-121b-4282-a178-edb50d60e716 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.085569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c43b7c-121b-4282-a178-edb50d60e716 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:48.087021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c43b7c-121b-4282-a178-edb50d60e716 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d1c43b7c-121b-4282-a178-edb50d60e716 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:48.098920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:48.099959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 12:07:48.100798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:48.106414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 384/773] 10.222.0.71 () {56 vars in 1129 bytes} [Wed Sep 21 12:07:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:48.107397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c43b7c-121b-4282-a178-edb50d60e716 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:48.175419 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245db4de-e6b2-4ff5-be95-c6a53e9f0d2a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:48.186300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:48.208950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:48.217284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d885c-df59-416e-876a-9d153da78e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:48.218307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d885c-df59-416e-876a-9d153da78e27 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.218825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d885c-df59-416e-876a-9d153da78e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.219323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d885c-df59-416e-876a-9d153da78e27 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.220717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d885c-df59-416e-876a-9d153da78e27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:48.221294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d885c-df59-416e-876a-9d153da78e27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:48.223415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d885c-df59-416e-876a-9d153da78e27 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:48.237150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 390/774] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:48.238046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d885c-df59-416e-876a-9d153da78e27 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:48.240483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c43b7c-121b-4282-a178-edb50d60e716 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:48.291580 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf2f12-982b-4023-a983-afcb607f30f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:48.292996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf2f12-982b-4023-a983-afcb607f30f5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.293494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf2f12-982b-4023-a983-afcb607f30f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.293978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf2f12-982b-4023-a983-afcb607f30f5 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.294837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf2f12-982b-4023-a983-afcb607f30f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:48.297304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf2f12-982b-4023-a983-afcb607f30f5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:48.304707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 385/775] 10.222.0.71 () {56 vars in 1087 bytes} [Wed Sep 21 12:07:48 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:48.305920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf2f12-982b-4023-a983-afcb607f30f5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:48.350621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d885c-df59-416e-876a-9d153da78e27 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:48.402353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabe4cd-b279-4425-931b-1f42faefd63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:48.402967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabe4cd-b279-4425-931b-1f42faefd63a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.403248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabe4cd-b279-4425-931b-1f42faefd63a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.403523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabe4cd-b279-4425-931b-1f42faefd63a None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.404995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabe4cd-b279-4425-931b-1f42faefd63a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:48.404995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabe4cd-b279-4425-931b-1f42faefd63a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:48.405951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabe4cd-b279-4425-931b-1f42faefd63a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:48.411712 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 391/776] 10.222.0.71 () {56 vars in 1124 bytes} [Wed Sep 21 12:07:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:48.417899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabe4cd-b279-4425-931b-1f42faefd63a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:48.445268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf2f12-982b-4023-a983-afcb607f30f5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:48.479364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:48.482446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:48.499294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65247e4f-0e72-432d-a445-6c566d4353dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:48.500486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65247e4f-0e72-432d-a445-6c566d4353dc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.500773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65247e4f-0e72-432d-a445-6c566d4353dc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.501051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65247e4f-0e72-432d-a445-6c566d4353dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.501796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65247e4f-0e72-432d-a445-6c566d4353dc None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=376fd55b948f49fda100a7da983f285c) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:48.502109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65247e4f-0e72-432d-a445-6c566d4353dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:48.503590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65247e4f-0e72-432d-a445-6c566d4353dc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:48.510869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 386/777] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:48 2022] GET /identity/v3/role_assignments?user.id=376fd55b948f49fda100a7da983f285c&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:48.538673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabe4cd-b279-4425-931b-1f42faefd63a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:48.582100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f0f78-4af6-423d-9fad-e44905d4a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:48.583027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f0f78-4af6-423d-9fad-e44905d4a45b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.583552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f0f78-4af6-423d-9fad-e44905d4a45b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.584000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f0f78-4af6-423d-9fad-e44905d4a45b None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.584824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f0f78-4af6-423d-9fad-e44905d4a45b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:48.585290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f0f78-4af6-423d-9fad-e44905d4a45b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:48.587042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f0f78-4af6-423d-9fad-e44905d4a45b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:48.594321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 392/778] 10.222.0.71 () {56 vars in 1124 bytes} [Wed Sep 21 12:07:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:48.600440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65247e4f-0e72-432d-a445-6c566d4353dc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:48.739129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65247e4f-0e72-432d-a445-6c566d4353dc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:48.776957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2c12a-c608-4382-961d-b7dc69b32172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:48.777578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2c12a-c608-4382-961d-b7dc69b32172 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.777860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2c12a-c608-4382-961d-b7dc69b32172 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.778132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2c12a-c608-4382-961d-b7dc69b32172 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.778838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2c12a-c608-4382-961d-b7dc69b32172 None admin] RBAC: Adding query filter params (interface=public, service_id=7dcb45f880fa463d8150784839d52dca, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:48.779147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2c12a-c608-4382-961d-b7dc69b32172 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:48.780602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2c12a-c608-4382-961d-b7dc69b32172 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:48.788991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 387/779] 10.222.0.71 () {56 vars in 1251 bytes} [Wed Sep 21 12:07:48 2022] GET /identity/v3/endpoints?service_id=7dcb45f880fa463d8150784839d52dca&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:48.794874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f0f78-4af6-423d-9fad-e44905d4a45b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:48.923590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37343e-77c3-4cce-8e29-981df06d0659 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.923976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37343e-77c3-4cce-8e29-981df06d0659 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.924153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37343e-77c3-4cce-8e29-981df06d0659 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.925659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 388/780] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:48.931263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d00a9-aad9-4622-9c69-41338a314034 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.931878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d00a9-aad9-4622-9c69-41338a314034 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.932360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d00a9-aad9-4622-9c69-41338a314034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.940232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f0f78-4af6-423d-9fad-e44905d4a45b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:48.951544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-835d00a9-aad9-4622-9c69-41338a314034 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:48.953127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-835d00a9-aad9-4622-9c69-41338a314034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:48.967791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:48.982201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3113b5-25ca-43bf-89aa-bd6653d68071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:48.983273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3113b5-25ca-43bf-89aa-bd6653d68071 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:48.983921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3113b5-25ca-43bf-89aa-bd6653d68071 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:48.984433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3113b5-25ca-43bf-89aa-bd6653d68071 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:48.985064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3113b5-25ca-43bf-89aa-bd6653d68071 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:48.986492 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3113b5-25ca-43bf-89aa-bd6653d68071 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:48.995781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 393/781] 10.222.0.71 () {56 vars in 1087 bytes} [Wed Sep 21 12:07:48 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:49.038776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:49.069601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 389/782] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:07:49.394458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a784b-c985-4fb8-a71c-79676cb9b5e6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:49.395315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a784b-c985-4fb8-a71c-79676cb9b5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:49.396084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a784b-c985-4fb8-a71c-79676cb9b5e6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:49.398268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 394/783] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:49.409790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff54afd-547e-4bd6-a9e2-1b341f8f87d2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:49.410085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff54afd-547e-4bd6-a9e2-1b341f8f87d2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:49.410431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff54afd-547e-4bd6-a9e2-1b341f8f87d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:49.438122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1ff54afd-547e-4bd6-a9e2-1b341f8f87d2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:49.439995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1ff54afd-547e-4bd6-a9e2-1b341f8f87d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:49.506243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a784b-c985-4fb8-a71c-79676cb9b5e6 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:49.518991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 390/784] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:07:49.661177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a784b-c985-4fb8-a71c-79676cb9b5e6 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:49.715340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d698e-67d9-4154-b93e-fa1d1b20be62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:49.716001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d698e-67d9-4154-b93e-fa1d1b20be62 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:49.716537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d698e-67d9-4154-b93e-fa1d1b20be62 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:49.720065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d698e-67d9-4154-b93e-fa1d1b20be62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:49.740741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d698e-67d9-4154-b93e-fa1d1b20be62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-669d698e-67d9-4154-b93e-fa1d1b20be62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:49.745776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:49.746694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:49.748651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 395/785] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:49.753142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff54afd-547e-4bd6-a9e2-1b341f8f87d2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:49.822760 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d698e-67d9-4154-b93e-fa1d1b20be62 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:49.901505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:49.930716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff54afd-547e-4bd6-a9e2-1b341f8f87d2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:49.968428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:49.991437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fad3a3-4f89-4fbd-90b9-0eeaf18ecaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:49.992072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fad3a3-4f89-4fbd-90b9-0eeaf18ecaac None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:49.992346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fad3a3-4f89-4fbd-90b9-0eeaf18ecaac None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:49.992612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fad3a3-4f89-4fbd-90b9-0eeaf18ecaac None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:49.993297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fad3a3-4f89-4fbd-90b9-0eeaf18ecaac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:49.993592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fad3a3-4f89-4fbd-90b9-0eeaf18ecaac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:49.994959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fad3a3-4f89-4fbd-90b9-0eeaf18ecaac None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:50.005999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d698e-67d9-4154-b93e-fa1d1b20be62 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:50.014116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 391/786] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:50.017328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fad3a3-4f89-4fbd-90b9-0eeaf18ecaac None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:50.070889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cede10-3292-4e1d-b722-abb55dd2e45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:50.071569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cede10-3292-4e1d-b722-abb55dd2e45f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:50.071867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cede10-3292-4e1d-b722-abb55dd2e45f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:50.072136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cede10-3292-4e1d-b722-abb55dd2e45f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:50.084566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cede10-3292-4e1d-b722-abb55dd2e45f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f3cede10-3292-4e1d-b722-abb55dd2e45f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:50.089392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:50.090320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:50.091150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:50.093576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 396/787] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:50.097857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cede10-3292-4e1d-b722-abb55dd2e45f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:50.212472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cede10-3292-4e1d-b722-abb55dd2e45f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:50.213974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:50.217467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fad3a3-4f89-4fbd-90b9-0eeaf18ecaac None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:50.280875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7a4c0-4c7e-4f1f-bcea-0382b45ca943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:50.281515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7a4c0-4c7e-4f1f-bcea-0382b45ca943 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:50.281797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7a4c0-4c7e-4f1f-bcea-0382b45ca943 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:50.282068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7a4c0-4c7e-4f1f-bcea-0382b45ca943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:50.283177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ccdc3-1c58-483f-aeb8-9ed62c565a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:50.283787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ccdc3-1c58-483f-aeb8-9ed62c565a91 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:50.284072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ccdc3-1c58-483f-aeb8-9ed62c565a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:50.284338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ccdc3-1c58-483f-aeb8-9ed62c565a91 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:50.285002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ccdc3-1c58-483f-aeb8-9ed62c565a91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:50.285311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ccdc3-1c58-483f-aeb8-9ed62c565a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:50.286686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ccdc3-1c58-483f-aeb8-9ed62c565a91 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:50.304229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 397/788] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:50.305004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7a4c0-4c7e-4f1f-bcea-0382b45ca943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:50.307084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7a4c0-4c7e-4f1f-bcea-0382b45ca943 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:50.310667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ccdc3-1c58-483f-aeb8-9ed62c565a91 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-60d7a4c0-4c7e-4f1f-bcea-0382b45ca943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:50.333449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:50.334475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:50.335368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:50.335368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:50.335368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:50.335910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 392/789] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:50.341555 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7a4c0-4c7e-4f1f-bcea-0382b45ca943 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:50.392131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:50.481076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ccdc3-1c58-483f-aeb8-9ed62c565a91 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:50.520032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7a4c0-4c7e-4f1f-bcea-0382b45ca943 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:50.545960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4cd03-f9dc-470b-b454-bbcb2ba695b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:50.546596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4cd03-f9dc-470b-b454-bbcb2ba695b6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:50.546869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4cd03-f9dc-470b-b454-bbcb2ba695b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:50.547137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4cd03-f9dc-470b-b454-bbcb2ba695b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:50.567638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4cd03-f9dc-470b-b454-bbcb2ba695b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:50.569936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4cd03-f9dc-470b-b454-bbcb2ba695b6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:50.583607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f8e12-d6d9-465b-8b85-17c9fb5a950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:50.584238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f8e12-d6d9-465b-8b85-17c9fb5a950c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:50.584503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f8e12-d6d9-465b-8b85-17c9fb5a950c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:50.584774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f8e12-d6d9-465b-8b85-17c9fb5a950c None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:50.585411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f8e12-d6d9-465b-8b85-17c9fb5a950c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:50.585707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f8e12-d6d9-465b-8b85-17c9fb5a950c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:50.587241 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f8e12-d6d9-465b-8b85-17c9fb5a950c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5cd4cd03-f9dc-470b-b454-bbcb2ba695b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:50.598831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:50.599797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:50.600696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:50.600696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:50.600696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:50.600696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 398/790] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:50.608759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4cd03-f9dc-470b-b454-bbcb2ba695b6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:50.612753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 393/791] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:50.626761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f8e12-d6d9-465b-8b85-17c9fb5a950c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:50.636628 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:50.753808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4cd03-f9dc-470b-b454-bbcb2ba695b6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:50.822483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622f2cc-483f-4683-85a0-b9ab4b52d164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:50.823424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622f2cc-483f-4683-85a0-b9ab4b52d164 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:50.823702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622f2cc-483f-4683-85a0-b9ab4b52d164 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:50.823993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622f2cc-483f-4683-85a0-b9ab4b52d164 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:50.824642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622f2cc-483f-4683-85a0-b9ab4b52d164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:50.824944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622f2cc-483f-4683-85a0-b9ab4b52d164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:50.826331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622f2cc-483f-4683-85a0-b9ab4b52d164 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:50.828161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:50.845140 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f8e12-d6d9-465b-8b85-17c9fb5a950c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:50.853774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 399/792] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:50.859034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622f2cc-483f-4683-85a0-b9ab4b52d164 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:50.911938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edec06-4011-49c1-8816-8b8d31ce08f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:50.912583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edec06-4011-49c1-8816-8b8d31ce08f0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:50.912863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edec06-4011-49c1-8816-8b8d31ce08f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:50.913138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edec06-4011-49c1-8816-8b8d31ce08f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:50.930954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edec06-4011-49c1-8816-8b8d31ce08f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:50.932482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edec06-4011-49c1-8816-8b8d31ce08f0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-65edec06-4011-49c1-8816-8b8d31ce08f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:50.955925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:50.956937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:50.957885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 394/793] 10.222.0.71 () {56 vars in 1093 bytes} [Wed Sep 21 12:07:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:50.970595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edec06-4011-49c1-8816-8b8d31ce08f0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:51.011977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:51.055270 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622f2cc-483f-4683-85a0-b9ab4b52d164 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:51.105589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e764de4-c62f-4966-b313-eade9df66fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:51.106343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e764de4-c62f-4966-b313-eade9df66fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.106687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e764de4-c62f-4966-b313-eade9df66fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.107334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e764de4-c62f-4966-b313-eade9df66fe5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.116669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e764de4-c62f-4966-b313-eade9df66fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:51.118737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e764de4-c62f-4966-b313-eade9df66fe5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:51.131431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edec06-4011-49c1-8816-8b8d31ce08f0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8e764de4-c62f-4966-b313-eade9df66fe5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:51.134779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:51.135943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:51.137107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:51.140644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 400/794] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:51.140644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb900d9-2c9e-435f-8e7d-720dcbdfd654 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.140644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb900d9-2c9e-435f-8e7d-720dcbdfd654 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.140644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb900d9-2c9e-435f-8e7d-720dcbdfd654 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.141276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 401/795] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:51.146106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb900d9-2c9e-435f-8e7d-720dcbdfd654 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:51.176064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6dc92-f76d-4b03-a78c-f72f2cf62bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:51.176811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6dc92-f76d-4b03-a78c-f72f2cf62bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.177098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6dc92-f76d-4b03-a78c-f72f2cf62bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.177416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6dc92-f76d-4b03-a78c-f72f2cf62bc9 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.178155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6dc92-f76d-4b03-a78c-f72f2cf62bc9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:51.178781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6dc92-f76d-4b03-a78c-f72f2cf62bc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:51.181016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6dc92-f76d-4b03-a78c-f72f2cf62bc9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:51.208025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 395/796] 10.222.0.71 () {56 vars in 1133 bytes} [Wed Sep 21 12:07:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:51.210211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab20f3-63e5-498d-8087-69ce6d0b0b5f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.210500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab20f3-63e5-498d-8087-69ce6d0b0b5f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.210881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab20f3-63e5-498d-8087-69ce6d0b0b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.235274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-51ab20f3-63e5-498d-8087-69ce6d0b0b5f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:51.236614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-51ab20f3-63e5-498d-8087-69ce6d0b0b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:51.238882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:51.258539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb900d9-2c9e-435f-8e7d-720dcbdfd654 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:51.301397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d6dc1-8b8c-44f4-9546-d9bbe6243afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:51.301945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d6dc1-8b8c-44f4-9546-d9bbe6243afc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.302224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d6dc1-8b8c-44f4-9546-d9bbe6243afc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.302497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d6dc1-8b8c-44f4-9546-d9bbe6243afc None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.303247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d6dc1-8b8c-44f4-9546-d9bbe6243afc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:51.303660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d6dc1-8b8c-44f4-9546-d9bbe6243afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:51.305303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d6dc1-8b8c-44f4-9546-d9bbe6243afc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:51.309916 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:51.327953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 396/797] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:07:51.328950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab20f3-63e5-498d-8087-69ce6d0b0b5f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:51.334071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 402/798] 10.222.0.71 () {56 vars in 1135 bytes} [Wed Sep 21 12:07:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:51.339045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d6dc1-8b8c-44f4-9546-d9bbe6243afc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:51.442609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab20f3-63e5-498d-8087-69ce6d0b0b5f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:51.485292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e6407-26fb-4fb3-b1d2-56ae0849313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:51.485910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e6407-26fb-4fb3-b1d2-56ae0849313e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.486191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e6407-26fb-4fb3-b1d2-56ae0849313e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.486465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e6407-26fb-4fb3-b1d2-56ae0849313e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.495515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:51.503223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d6dc1-8b8c-44f4-9546-d9bbe6243afc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:51.546364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742d9c0-6131-4c16-ab2a-dcefd75cc8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:51.547174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742d9c0-6131-4c16-ab2a-dcefd75cc8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.547554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742d9c0-6131-4c16-ab2a-dcefd75cc8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.547981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742d9c0-6131-4c16-ab2a-dcefd75cc8b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.560783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742d9c0-6131-4c16-ab2a-dcefd75cc8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:51.562498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742d9c0-6131-4c16-ab2a-dcefd75cc8b5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:51.564247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e6407-26fb-4fb3-b1d2-56ae0849313e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:51.565823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e6407-26fb-4fb3-b1d2-56ae0849313e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8742d9c0-6131-4c16-ab2a-dcefd75cc8b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:51.581061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:51.582292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:51.583332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:51.583332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:51.583332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:51.583858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 403/799] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:51.585097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cf373-7daf-40b3-92e5-0ace852556ec None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.587735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cf373-7daf-40b3-92e5-0ace852556ec None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.587735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cf373-7daf-40b3-92e5-0ace852556ec None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.588119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 404/800] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:51.589509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cf373-7daf-40b3-92e5-0ace852556ec None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:51.643719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:51.679017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 397/801] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:07:51.680438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ca675-f0f6-4d8d-ac81-2fce46f5a5f1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.680762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ca675-f0f6-4d8d-ac81-2fce46f5a5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.681119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ca675-f0f6-4d8d-ac81-2fce46f5a5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.699451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-090ca675-f0f6-4d8d-ac81-2fce46f5a5f1 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:51.700678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-090ca675-f0f6-4d8d-ac81-2fce46f5a5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:51.705346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cf373-7daf-40b3-92e5-0ace852556ec None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:51.731338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:51.740265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048805f-57ac-494c-8317-c6fdbf497ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:51.740873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048805f-57ac-494c-8317-c6fdbf497ddb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.741154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048805f-57ac-494c-8317-c6fdbf497ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.741428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048805f-57ac-494c-8317-c6fdbf497ddb None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.742076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048805f-57ac-494c-8317-c6fdbf497ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:51.742385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048805f-57ac-494c-8317-c6fdbf497ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:51.743813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048805f-57ac-494c-8317-c6fdbf497ddb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:51.756747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 405/802] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:51.758425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f410947-21cb-471e-bfeb-9ce84708a5a2 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.758712 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f410947-21cb-471e-bfeb-9ce84708a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.759500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f410947-21cb-471e-bfeb-9ce84708a5a2 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.760773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 406/803] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:51.761669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f410947-21cb-471e-bfeb-9ce84708a5a2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:51.777266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 398/804] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:07:51.778122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ca675-f0f6-4d8d-ac81-2fce46f5a5f1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:51.876761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f410947-21cb-471e-bfeb-9ce84708a5a2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:51.885169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:51.903590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ca675-f0f6-4d8d-ac81-2fce46f5a5f1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:51.916505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d38ff8-f3a1-409c-96dc-b1304e0db418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:51.917450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38ff8-f3a1-409c-96dc-b1304e0db418 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.917941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38ff8-f3a1-409c-96dc-b1304e0db418 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.918393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d38ff8-f3a1-409c-96dc-b1304e0db418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.929254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d38ff8-f3a1-409c-96dc-b1304e0db418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:51.931517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d38ff8-f3a1-409c-96dc-b1304e0db418 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:51.941374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fa5f9-6481-4a72-b440-74f284e74672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:51.942413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fa5f9-6481-4a72-b440-74f284e74672 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.942708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fa5f9-6481-4a72-b440-74f284e74672 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.942981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fa5f9-6481-4a72-b440-74f284e74672 None admin] PATH_INFO: `/v3/services/network` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.943570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fa5f9-6481-4a72-b440-74f284e74672 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:51.944983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fa5f9-6481-4a72-b440-74f284e74672 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-b2d38ff8-f3a1-409c-96dc-b1304e0db418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:51.948936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:51.949886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:51.950787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:51.950787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:51.950787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:51.950787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 407/805] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:51.951038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d38ff8-f3a1-409c-96dc-b1304e0db418 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0e1fa5f9-6481-4a72-b440-74f284e74672 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:51.962019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:51.963040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 12:07:51.963928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:51.968656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 399/806] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:51 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:51.970998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0db4ed-5efa-47f6-8b68-cf8c62f03c12 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:51.971486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0db4ed-5efa-47f6-8b68-cf8c62f03c12 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:51.971798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0db4ed-5efa-47f6-8b68-cf8c62f03c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:51.996143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6d0db4ed-5efa-47f6-8b68-cf8c62f03c12 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:51.996143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6d0db4ed-5efa-47f6-8b68-cf8c62f03c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:52.026515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:52.075625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d38ff8-f3a1-409c-96dc-b1304e0db418 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:52.083807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 400/807] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:07:52.083807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0db4ed-5efa-47f6-8b68-cf8c62f03c12 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.120054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c8398-d9b7-485a-9baf-6c71af2355fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:52.120566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c8398-d9b7-485a-9baf-6c71af2355fc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:52.120944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c8398-d9b7-485a-9baf-6c71af2355fc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:52.121218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c8398-d9b7-485a-9baf-6c71af2355fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:52.128973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c8398-d9b7-485a-9baf-6c71af2355fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:52.130348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c8398-d9b7-485a-9baf-6c71af2355fc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6e7c8398-d9b7-485a-9baf-6c71af2355fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:52.143497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:52.151213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 408/808] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:52.153108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c8398-d9b7-485a-9baf-6c71af2355fc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.173845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:52.206005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0db4ed-5efa-47f6-8b68-cf8c62f03c12 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:52.253983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30cbde-9ace-44b8-987d-aadf07299bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:52.254592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30cbde-9ace-44b8-987d-aadf07299bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:52.254875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30cbde-9ace-44b8-987d-aadf07299bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:52.255148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30cbde-9ace-44b8-987d-aadf07299bf0 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:52.255822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30cbde-9ace-44b8-987d-aadf07299bf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:52.256126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30cbde-9ace-44b8-987d-aadf07299bf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:52.257497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30cbde-9ace-44b8-987d-aadf07299bf0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:52.277999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 401/809] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:52.278863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30cbde-9ace-44b8-987d-aadf07299bf0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.283877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c8398-d9b7-485a-9baf-6c71af2355fc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:52.313554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:52.322773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4793d8-1748-4af1-ab16-f281c16cfec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:52.323378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4793d8-1748-4af1-ab16-f281c16cfec6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:52.323654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4793d8-1748-4af1-ab16-f281c16cfec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:52.325223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4793d8-1748-4af1-ab16-f281c16cfec6 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:52.325223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4793d8-1748-4af1-ab16-f281c16cfec6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:52.325223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4793d8-1748-4af1-ab16-f281c16cfec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:52.326572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4793d8-1748-4af1-ab16-f281c16cfec6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:52.335590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 409/810] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:07:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:52.335590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4793d8-1748-4af1-ab16-f281c16cfec6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.405709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30cbde-9ace-44b8-987d-aadf07299bf0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:52.428526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:52.442138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f703a1-f9e4-4f81-83e1-7b431c1dc101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:52.442769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f703a1-f9e4-4f81-83e1-7b431c1dc101 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:52.443051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f703a1-f9e4-4f81-83e1-7b431c1dc101 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:52.443334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f703a1-f9e4-4f81-83e1-7b431c1dc101 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:52.450315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4793d8-1748-4af1-ab16-f281c16cfec6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:52.454232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f703a1-f9e4-4f81-83e1-7b431c1dc101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a3f703a1-f9e4-4f81-83e1-7b431c1dc101 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:52.459638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:52.460888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 402/811] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:52.461930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f703a1-f9e4-4f81-83e1-7b431c1dc101 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.487425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bc94a-ac29-40e2-82c3-396b88d619a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:52.488068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bc94a-ac29-40e2-82c3-396b88d619a6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:52.488354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bc94a-ac29-40e2-82c3-396b88d619a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:52.488622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20bc94a-ac29-40e2-82c3-396b88d619a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:52.489313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bc94a-ac29-40e2-82c3-396b88d619a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:52.489624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bc94a-ac29-40e2-82c3-396b88d619a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:52.491062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20bc94a-ac29-40e2-82c3-396b88d619a6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:52.503532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 410/812] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:52.504398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bc94a-ac29-40e2-82c3-396b88d619a6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.549429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:52.568840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f703a1-f9e4-4f81-83e1-7b431c1dc101 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:52.602314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389bcb1-01cf-4b35-9500-eb4822b1857e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:52.602945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389bcb1-01cf-4b35-9500-eb4822b1857e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:52.603223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389bcb1-01cf-4b35-9500-eb4822b1857e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:52.603492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389bcb1-01cf-4b35-9500-eb4822b1857e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:52.603684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20bc94a-ac29-40e2-82c3-396b88d619a6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:52.610218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389bcb1-01cf-4b35-9500-eb4822b1857e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:52.611661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389bcb1-01cf-4b35-9500-eb4822b1857e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a389bcb1-01cf-4b35-9500-eb4822b1857e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:52.623901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:52.625494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 403/813] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:52.626392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389bcb1-01cf-4b35-9500-eb4822b1857e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.646995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba16a9e-0962-4027-b7c8-e59828facf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:52.647612 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba16a9e-0962-4027-b7c8-e59828facf9e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:52.647924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba16a9e-0962-4027-b7c8-e59828facf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:52.648201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba16a9e-0962-4027-b7c8-e59828facf9e None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:52.648835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba16a9e-0962-4027-b7c8-e59828facf9e None admin] RBAC: Adding query filter params (type=network) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:52.649219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba16a9e-0962-4027-b7c8-e59828facf9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:52.650532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba16a9e-0962-4027-b7c8-e59828facf9e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:52.658081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 411/814] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:07:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:52.659587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba16a9e-0962-4027-b7c8-e59828facf9e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.674181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:52.766479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389bcb1-01cf-4b35-9500-eb4822b1857e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:52.802633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc1ed7-b2cf-4398-8c57-25dc9b5d9345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:52.803275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc1ed7-b2cf-4398-8c57-25dc9b5d9345 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:52.803480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba16a9e-0962-4027-b7c8-e59828facf9e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:52.803586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc1ed7-b2cf-4398-8c57-25dc9b5d9345 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:52.803843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc1ed7-b2cf-4398-8c57-25dc9b5d9345 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:52.804436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc1ed7-b2cf-4398-8c57-25dc9b5d9345 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:52.806352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc1ed7-b2cf-4398-8c57-25dc9b5d9345 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-4fcc1ed7-b2cf-4398-8c57-25dc9b5d9345 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:52.815412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:52.816754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:52.817945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 404/815] 10.222.0.71 () {56 vars in 1129 bytes} [Wed Sep 21 12:07:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:52.818783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc1ed7-b2cf-4398-8c57-25dc9b5d9345 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.830232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:52.861264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803330c-3085-4d94-8716-1b64d16a46dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:52.862253 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803330c-3085-4d94-8716-1b64d16a46dc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:52.865187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803330c-3085-4d94-8716-1b64d16a46dc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:52.865187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803330c-3085-4d94-8716-1b64d16a46dc None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:52.865187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803330c-3085-4d94-8716-1b64d16a46dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:52.865187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803330c-3085-4d94-8716-1b64d16a46dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:52.867179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803330c-3085-4d94-8716-1b64d16a46dc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:52.880670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 412/816] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:52.881852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803330c-3085-4d94-8716-1b64d16a46dc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:52.965112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:53.004040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803330c-3085-4d94-8716-1b64d16a46dc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.037644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b394a49-0474-470b-b69d-ed0cfbb99395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.038245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b394a49-0474-470b-b69d-ed0cfbb99395 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.038518 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b394a49-0474-470b-b69d-ed0cfbb99395 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.038783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b394a49-0474-470b-b69d-ed0cfbb99395 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.039459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b394a49-0474-470b-b69d-ed0cfbb99395 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:53.039787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b394a49-0474-470b-b69d-ed0cfbb99395 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.041356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc1ed7-b2cf-4398-8c57-25dc9b5d9345 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.041800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b394a49-0474-470b-b69d-ed0cfbb99395 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.055130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 413/817] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:53.056569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b394a49-0474-470b-b69d-ed0cfbb99395 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:53.076648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a57f8-a7b5-42d0-ba1c-2b86a0f122de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.077316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a57f8-a7b5-42d0-ba1c-2b86a0f122de None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.077596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a57f8-a7b5-42d0-ba1c-2b86a0f122de None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.077867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a57f8-a7b5-42d0-ba1c-2b86a0f122de None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/e0bf75c730c946bc9168dd418e7af637/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.080253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a57f8-a7b5-42d0-ba1c-2b86a0f122de None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=e0bf75c730c946bc9168dd418e7af637, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.081690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a57f8-a7b5-42d0-ba1c-2b86a0f122de None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.095585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 405/818] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:07:52 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/e0bf75c730c946bc9168dd418e7af637/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:53.096456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a57f8-a7b5-42d0-ba1c-2b86a0f122de None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:53.178539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b394a49-0474-470b-b69d-ed0cfbb99395 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.191182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:53.208320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:53.221197 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1dd07-f5b9-4ab8-9690-81b05ff1afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.222281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1dd07-f5b9-4ab8-9690-81b05ff1afdb None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.222563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1dd07-f5b9-4ab8-9690-81b05ff1afdb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.223008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1dd07-f5b9-4ab8-9690-81b05ff1afdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.223702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1dd07-f5b9-4ab8-9690-81b05ff1afdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.225155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1dd07-f5b9-4ab8-9690-81b05ff1afdb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.240623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a57f8-a7b5-42d0-ba1c-2b86a0f122de None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.251816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 414/819] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:07:53 2022] POST /identity/v3/endpoints => generated 337 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:53.252919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1dd07-f5b9-4ab8-9690-81b05ff1afdb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:53.312156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822ec27-fc2a-4600-9e26-3791108466fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.313851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822ec27-fc2a-4600-9e26-3791108466fb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.314201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822ec27-fc2a-4600-9e26-3791108466fb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.314471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5822ec27-fc2a-4600-9e26-3791108466fb None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.315115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822ec27-fc2a-4600-9e26-3791108466fb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:53.315537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822ec27-fc2a-4600-9e26-3791108466fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.316998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5822ec27-fc2a-4600-9e26-3791108466fb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.322763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 406/820] 10.222.0.71 () {56 vars in 1124 bytes} [Wed Sep 21 12:07:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:53.323472 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822ec27-fc2a-4600-9e26-3791108466fb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:53.396163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1dd07-f5b9-4ab8-9690-81b05ff1afdb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.446127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635a10f-d64e-4cd9-a78b-1c0515914e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.448368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635a10f-d64e-4cd9-a78b-1c0515914e23 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.448995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635a10f-d64e-4cd9-a78b-1c0515914e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.449548 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635a10f-d64e-4cd9-a78b-1c0515914e23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.466110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635a10f-d64e-4cd9-a78b-1c0515914e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.467712 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635a10f-d64e-4cd9-a78b-1c0515914e23 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.481347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5822ec27-fc2a-4600-9e26-3791108466fb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-9635a10f-d64e-4cd9-a78b-1c0515914e23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:53.487041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:53.488309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:53.489239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:53.489239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:53.489239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:53.489239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 415/821] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:53.489239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635a10f-d64e-4cd9-a78b-1c0515914e23 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:53.517858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:53.528084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde61be8-2646-49eb-8cc5-cf19edb12cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.528617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde61be8-2646-49eb-8cc5-cf19edb12cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.528896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde61be8-2646-49eb-8cc5-cf19edb12cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.529165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde61be8-2646-49eb-8cc5-cf19edb12cc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.530294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde61be8-2646-49eb-8cc5-cf19edb12cc5 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=463487449c1d4aada1b5f279c959b0e6) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:53.530294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde61be8-2646-49eb-8cc5-cf19edb12cc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.531670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde61be8-2646-49eb-8cc5-cf19edb12cc5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.538563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 407/822] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:53 2022] GET /identity/v3/role_assignments?user.id=463487449c1d4aada1b5f279c959b0e6&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:53.539450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde61be8-2646-49eb-8cc5-cf19edb12cc5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:53.556658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:53.637601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635a10f-d64e-4cd9-a78b-1c0515914e23 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.667726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde61be8-2646-49eb-8cc5-cf19edb12cc5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.676972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4f90e-5eb2-49aa-b457-179262c53185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.677244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4f90e-5eb2-49aa-b457-179262c53185 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.677482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4f90e-5eb2-49aa-b457-179262c53185 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.677762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4f90e-5eb2-49aa-b457-179262c53185 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.678420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4f90e-5eb2-49aa-b457-179262c53185 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:53.678728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4f90e-5eb2-49aa-b457-179262c53185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.680189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc4f90e-5eb2-49aa-b457-179262c53185 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.686730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 416/823] 10.222.0.71 () {56 vars in 1124 bytes} [Wed Sep 21 12:07:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:53.692589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4f90e-5eb2-49aa-b457-179262c53185 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:53.711804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a41d0-8ecd-4533-8bf7-ea3468a41ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.712813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a41d0-8ecd-4533-8bf7-ea3468a41ffd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.713436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a41d0-8ecd-4533-8bf7-ea3468a41ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.713909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a41d0-8ecd-4533-8bf7-ea3468a41ffd None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.714889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a41d0-8ecd-4533-8bf7-ea3468a41ffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:53.715413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a41d0-8ecd-4533-8bf7-ea3468a41ffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.717320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a41d0-8ecd-4533-8bf7-ea3468a41ffd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.732571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 408/824] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:53.738395 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a41d0-8ecd-4533-8bf7-ea3468a41ffd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:53.796615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:53.822887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:53.832215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4f90e-5eb2-49aa-b457-179262c53185 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.875536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584279a-22db-41a9-a413-f7d3d665f9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.876183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584279a-22db-41a9-a413-f7d3d665f9ad None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.876465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584279a-22db-41a9-a413-f7d3d665f9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.877627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584279a-22db-41a9-a413-f7d3d665f9ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.877627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584279a-22db-41a9-a413-f7d3d665f9ad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.878664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a41d0-8ecd-4533-8bf7-ea3468a41ffd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:53.878770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584279a-22db-41a9-a413-f7d3d665f9ad None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.905824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 417/825] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:07:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:07:53.938878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999ad3-981b-4158-b0d5-34c8d89954ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:53.939955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999ad3-981b-4158-b0d5-34c8d89954ad None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:53.940494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999ad3-981b-4158-b0d5-34c8d89954ad None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:53.940984 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76999ad3-981b-4158-b0d5-34c8d89954ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:53.950412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76999ad3-981b-4158-b0d5-34c8d89954ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:53.952379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76999ad3-981b-4158-b0d5-34c8d89954ad None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-76999ad3-981b-4158-b0d5-34c8d89954ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:53.966729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:53.967829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:07:53.968822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:53.971628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 409/826] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:07:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:53.980629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584279a-22db-41a9-a413-f7d3d665f9ad None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:54.101355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584279a-22db-41a9-a413-f7d3d665f9ad None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:54.137576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ead7f-9c4a-4534-98e8-cc8c1e4a0d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:54.138134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ead7f-9c4a-4534-98e8-cc8c1e4a0d04 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:54.138413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ead7f-9c4a-4534-98e8-cc8c1e4a0d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:54.138686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ead7f-9c4a-4534-98e8-cc8c1e4a0d04 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:54.139427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ead7f-9c4a-4534-98e8-cc8c1e4a0d04 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:54.139729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ead7f-9c4a-4534-98e8-cc8c1e4a0d04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:54.141230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ead7f-9c4a-4534-98e8-cc8c1e4a0d04 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:54.161234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 418/827] 10.222.0.71 () {56 vars in 1149 bytes} [Wed Sep 21 12:07:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:54.167847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999ad3-981b-4158-b0d5-34c8d89954ad None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:54.292933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76999ad3-981b-4158-b0d5-34c8d89954ad None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:54.300645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:54.328888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23aa5a3-8fe6-4090-9bb2-a7dd709674d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:54.329519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23aa5a3-8fe6-4090-9bb2-a7dd709674d6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:54.329794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23aa5a3-8fe6-4090-9bb2-a7dd709674d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:54.330058 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23aa5a3-8fe6-4090-9bb2-a7dd709674d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:54.341631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23aa5a3-8fe6-4090-9bb2-a7dd709674d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:54.343075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23aa5a3-8fe6-4090-9bb2-a7dd709674d6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f23aa5a3-8fe6-4090-9bb2-a7dd709674d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:54.357982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:54.358929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:54.359842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:54.359842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:54.359842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:54.359842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 410/828] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:54.365535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ead7f-9c4a-4534-98e8-cc8c1e4a0d04 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:54.383411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:54.597417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ead7f-9c4a-4534-98e8-cc8c1e4a0d04 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:54.632764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ce3fa-95e7-4b7b-9e49-5bbade1de9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:54.633282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ce3fa-95e7-4b7b-9e49-5bbade1de9fa None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:54.633560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ce3fa-95e7-4b7b-9e49-5bbade1de9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:54.634360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ce3fa-95e7-4b7b-9e49-5bbade1de9fa None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:54.635006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ce3fa-95e7-4b7b-9e49-5bbade1de9fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:54.635309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ce3fa-95e7-4b7b-9e49-5bbade1de9fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:54.636715 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ce3fa-95e7-4b7b-9e49-5bbade1de9fa None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:54.650119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 419/829] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:54.655364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23aa5a3-8fe6-4090-9bb2-a7dd709674d6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:54.768410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23aa5a3-8fe6-4090-9bb2-a7dd709674d6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:54.803035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7b0a5-3c00-41b6-ba12-b49761a49cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:54.804089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7b0a5-3c00-41b6-ba12-b49761a49cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:54.804650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7b0a5-3c00-41b6-ba12-b49761a49cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:54.805120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c7b0a5-3c00-41b6-ba12-b49761a49cd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:54.812792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7b0a5-3c00-41b6-ba12-b49761a49cd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:54.814454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c7b0a5-3c00-41b6-ba12-b49761a49cd1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-64c7b0a5-3c00-41b6-ba12-b49761a49cd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:54.826701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:54.827677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:54.828588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:54.828588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:54.828588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:54.828588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:54.828588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:54.828588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:54.828588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:54.828588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:54.830918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 411/830] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:54.836329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ce3fa-95e7-4b7b-9e49-5bbade1de9fa None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:54.924459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:54.951536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ce3fa-95e7-4b7b-9e49-5bbade1de9fa None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:54.987400 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905db1b-744b-420a-9873-a99ef0fbae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:54.988046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905db1b-744b-420a-9873-a99ef0fbae92 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:54.988328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905db1b-744b-420a-9873-a99ef0fbae92 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:54.988601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905db1b-744b-420a-9873-a99ef0fbae92 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:54.989296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905db1b-744b-420a-9873-a99ef0fbae92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:54.989602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905db1b-744b-420a-9873-a99ef0fbae92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:54.991030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905db1b-744b-420a-9873-a99ef0fbae92 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:55.003834 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 420/831] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:55.009416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7b0a5-3c00-41b6-ba12-b49761a49cd1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:55.036028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:55.122963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c7b0a5-3c00-41b6-ba12-b49761a49cd1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:55.158487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f48b87-93a6-496a-8e92-ed8f67b215db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:55.159531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f48b87-93a6-496a-8e92-ed8f67b215db None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:55.160056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f48b87-93a6-496a-8e92-ed8f67b215db None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:55.160553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f48b87-93a6-496a-8e92-ed8f67b215db None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/376fd55b948f49fda100a7da983f285c/roles/fcc43e29f89e4c6cb1690a36d91ec4c1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:55.176497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f48b87-93a6-496a-8e92-ed8f67b215db None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=376fd55b948f49fda100a7da983f285c, role_id=fcc43e29f89e4c6cb1690a36d91ec4c1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:55.178742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f48b87-93a6-496a-8e92-ed8f67b215db None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:55.194387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 412/832] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:07:55 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/376fd55b948f49fda100a7da983f285c/roles/fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:07:55.373102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d955e-b871-4fb8-8bc2-cb528f4de5d8 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:55.373639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d955e-b871-4fb8-8bc2-cb528f4de5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:55.373639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d955e-b871-4fb8-8bc2-cb528f4de5d8 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:55.374980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 421/833] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:55.380682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857151a5-b136-4b23-9eb6-201ea9410b27 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:55.380972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857151a5-b136-4b23-9eb6-201ea9410b27 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:55.381251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857151a5-b136-4b23-9eb6-201ea9410b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:55.401731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-857151a5-b136-4b23-9eb6-201ea9410b27 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:55.403227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-857151a5-b136-4b23-9eb6-201ea9410b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:55.469419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:55.476506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 413/834] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:07:55.752450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d955e-b871-4fb8-8bc2-cb528f4de5d8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:55.869268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d955e-b871-4fb8-8bc2-cb528f4de5d8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:55.906303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47aaa1f-5fd9-49f9-9ea5-aa602deb7ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:55.906950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aaa1f-5fd9-49f9-9ea5-aa602deb7ece None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:55.907936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aaa1f-5fd9-49f9-9ea5-aa602deb7ece None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:55.908218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47aaa1f-5fd9-49f9-9ea5-aa602deb7ece None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:55.910443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:55.922352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47aaa1f-5fd9-49f9-9ea5-aa602deb7ece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c47aaa1f-5fd9-49f9-9ea5-aa602deb7ece None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:55.928164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:55.929154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:55.929976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 422/835] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:55.937933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857151a5-b136-4b23-9eb6-201ea9410b27 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:56.046224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857151a5-b136-4b23-9eb6-201ea9410b27 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:56.052955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e30b67-8c19-48be-a299-f8868f541e7b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.053328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e30b67-8c19-48be-a299-f8868f541e7b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.053516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e30b67-8c19-48be-a299-f8868f541e7b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.055243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 423/836] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:56.061079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f28e90-a42f-49e4-b46f-6d77f75a7462 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.061368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f28e90-a42f-49e4-b46f-6d77f75a7462 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.061754 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f28e90-a42f-49e4-b46f-6d77f75a7462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.081613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-64f28e90-a42f-49e4-b46f-6d77f75a7462 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:56.082851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64f28e90-a42f-49e4-b46f-6d77f75a7462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:56.084617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34930cfa-559b-43f7-8dcc-a4c1c8ee42eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:56.085432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34930cfa-559b-43f7-8dcc-a4c1c8ee42eb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.085746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34930cfa-559b-43f7-8dcc-a4c1c8ee42eb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.086193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34930cfa-559b-43f7-8dcc-a4c1c8ee42eb None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.086966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34930cfa-559b-43f7-8dcc-a4c1c8ee42eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:56.087298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34930cfa-559b-43f7-8dcc-a4c1c8ee42eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:56.088837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34930cfa-559b-43f7-8dcc-a4c1c8ee42eb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:56.098673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 414/837] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:07:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:56.106683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34930cfa-559b-43f7-8dcc-a4c1c8ee42eb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:56.172534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 424/838] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:07:56.181761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:56.228066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34930cfa-559b-43f7-8dcc-a4c1c8ee42eb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:56.246237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524e69b-bafb-487c-8b8a-07df165d717d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.246546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524e69b-bafb-487c-8b8a-07df165d717d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.246797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524e69b-bafb-487c-8b8a-07df165d717d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.248414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 425/839] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:56.253307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610c102-ef88-4c0b-a118-c5233fe4a9e5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.253585 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610c102-ef88-4c0b-a118-c5233fe4a9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.253864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610c102-ef88-4c0b-a118-c5233fe4a9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.267909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09748986-eebd-49df-8301-3847de36134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:56.269098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09748986-eebd-49df-8301-3847de36134b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.269575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09748986-eebd-49df-8301-3847de36134b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.270054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09748986-eebd-49df-8301-3847de36134b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.272291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3610c102-ef88-4c0b-a118-c5233fe4a9e5 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:56.273476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3610c102-ef88-4c0b-a118-c5233fe4a9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:56.281988 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09748986-eebd-49df-8301-3847de36134b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:56.283696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09748986-eebd-49df-8301-3847de36134b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-09748986-eebd-49df-8301-3847de36134b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:56.301011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:56.301979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:56.302887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:56.302887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:56.302887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:56.302887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 415/840] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:56.308569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09748986-eebd-49df-8301-3847de36134b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:56.358768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 426/841] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:07:56.416916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09748986-eebd-49df-8301-3847de36134b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:56.429632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:56.451711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e018-1a8e-41a2-8328-e4b965c22d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:56.452389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e018-1a8e-41a2-8328-e4b965c22d26 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.452662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e018-1a8e-41a2-8328-e4b965c22d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.452924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e018-1a8e-41a2-8328-e4b965c22d26 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.453559 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e018-1a8e-41a2-8328-e4b965c22d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:56.453852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e018-1a8e-41a2-8328-e4b965c22d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:56.455293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e018-1a8e-41a2-8328-e4b965c22d26 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:56.457195 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610c102-ef88-4c0b-a118-c5233fe4a9e5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:56.468072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 416/842] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:56.473210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e018-1a8e-41a2-8328-e4b965c22d26 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:56.529871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:56.586879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e018-1a8e-41a2-8328-e4b965c22d26 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:56.588920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610c102-ef88-4c0b-a118-c5233fe4a9e5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:56.628006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87403f04-43d4-4f57-bf38-09958dd057dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:56.629030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87403f04-43d4-4f57-bf38-09958dd057dc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.629493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87403f04-43d4-4f57-bf38-09958dd057dc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.629962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87403f04-43d4-4f57-bf38-09958dd057dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.638850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844e159-0e8d-4311-9f24-498597336cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:56.639927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844e159-0e8d-4311-9f24-498597336cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.640207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844e159-0e8d-4311-9f24-498597336cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.640476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844e159-0e8d-4311-9f24-498597336cf5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.640920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87403f04-43d4-4f57-bf38-09958dd057dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:56.641215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844e159-0e8d-4311-9f24-498597336cf5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:56.642687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844e159-0e8d-4311-9f24-498597336cf5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:56.643103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87403f04-43d4-4f57-bf38-09958dd057dc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0844e159-0e8d-4311-9f24-498597336cf5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:56.653412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:56.654495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 12:07:56.655373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:56.663874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 427/843] 10.222.0.71 () {56 vars in 1132 bytes} [Wed Sep 21 12:07:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-87403f04-43d4-4f57-bf38-09958dd057dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:56.664033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:56.665622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 12:07:56.666747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:56.667499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844e159-0e8d-4311-9f24-498597336cf5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:56.667628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 417/844] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:07:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:56.670014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87403f04-43d4-4f57-bf38-09958dd057dc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:56.706434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:56.767367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844e159-0e8d-4311-9f24-498597336cf5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:56.785229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:56.786907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87403f04-43d4-4f57-bf38-09958dd057dc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:56.805224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81879646-a59c-4286-8ce6-9daf8d1e03dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:56.806394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81879646-a59c-4286-8ce6-9daf8d1e03dc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.806667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81879646-a59c-4286-8ce6-9daf8d1e03dc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.806935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81879646-a59c-4286-8ce6-9daf8d1e03dc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.807521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81879646-a59c-4286-8ce6-9daf8d1e03dc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:56.808910 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81879646-a59c-4286-8ce6-9daf8d1e03dc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-81879646-a59c-4286-8ce6-9daf8d1e03dc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:56.817836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:56.819028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:56.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 428/845] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:07:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:56.820967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81879646-a59c-4286-8ce6-9daf8d1e03dc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:56.833182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36594378-7c69-4929-a33f-7ecbfe7c042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:56.834146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36594378-7c69-4929-a33f-7ecbfe7c042f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.834703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36594378-7c69-4929-a33f-7ecbfe7c042f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.835137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36594378-7c69-4929-a33f-7ecbfe7c042f None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.835968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36594378-7c69-4929-a33f-7ecbfe7c042f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:56.836275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36594378-7c69-4929-a33f-7ecbfe7c042f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:56.837735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36594378-7c69-4929-a33f-7ecbfe7c042f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:56.853008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 418/846] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:07:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:56.853008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36594378-7c69-4929-a33f-7ecbfe7c042f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:56.939091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81879646-a59c-4286-8ce6-9daf8d1e03dc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:56.954666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:56.962496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36594378-7c69-4929-a33f-7ecbfe7c042f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:56.974759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485130b-e9e3-4c10-baa3-b2ee99fc8912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:56.975369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485130b-e9e3-4c10-baa3-b2ee99fc8912 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:56.975651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485130b-e9e3-4c10-baa3-b2ee99fc8912 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:56.975956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485130b-e9e3-4c10-baa3-b2ee99fc8912 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:56.977015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485130b-e9e3-4c10-baa3-b2ee99fc8912 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:56.977839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485130b-e9e3-4c10-baa3-b2ee99fc8912 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:56.979282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485130b-e9e3-4c10-baa3-b2ee99fc8912 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:56.997299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 429/847] 10.222.0.71 () {56 vars in 1135 bytes} [Wed Sep 21 12:07:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:56.998135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485130b-e9e3-4c10-baa3-b2ee99fc8912 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:57.007990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6522631-6148-4511-b196-2fd6faece8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:57.008630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6522631-6148-4511-b196-2fd6faece8fd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:57.008913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6522631-6148-4511-b196-2fd6faece8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:57.009187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6522631-6148-4511-b196-2fd6faece8fd None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:57.010021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6522631-6148-4511-b196-2fd6faece8fd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:57.010127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6522631-6148-4511-b196-2fd6faece8fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:57.011452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6522631-6148-4511-b196-2fd6faece8fd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:57.021274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 419/848] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:07:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:57.021274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6522631-6148-4511-b196-2fd6faece8fd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:57.042493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:57.104052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485130b-e9e3-4c10-baa3-b2ee99fc8912 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:57.131340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6522631-6148-4511-b196-2fd6faece8fd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:57.138863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2272f-ea63-4052-8f7b-a7f4faed07a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:57.139478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2272f-ea63-4052-8f7b-a7f4faed07a5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:57.139778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2272f-ea63-4052-8f7b-a7f4faed07a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:57.140168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2272f-ea63-4052-8f7b-a7f4faed07a5 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:57.140801 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2272f-ea63-4052-8f7b-a7f4faed07a5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:57.141102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2272f-ea63-4052-8f7b-a7f4faed07a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:57.142467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2272f-ea63-4052-8f7b-a7f4faed07a5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:57.147209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 430/849] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:07:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:57.148035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2272f-ea63-4052-8f7b-a7f4faed07a5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:57.181016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0b02a-f60e-43f1-947f-055b5116df13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:57.181841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0b02a-f60e-43f1-947f-055b5116df13 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:57.182406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0b02a-f60e-43f1-947f-055b5116df13 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:57.182406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0b02a-f60e-43f1-947f-055b5116df13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:57.219558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:57.278679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2272f-ea63-4052-8f7b-a7f4faed07a5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:57.279682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0b02a-f60e-43f1-947f-055b5116df13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:57.282228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:57.283913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0b02a-f60e-43f1-947f-055b5116df13 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:57.320993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491beed0-a617-40ec-8db1-a92c7030d558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:57.321633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491beed0-a617-40ec-8db1-a92c7030d558 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:57.321943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491beed0-a617-40ec-8db1-a92c7030d558 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:57.322363 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491beed0-a617-40ec-8db1-a92c7030d558 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:57.323103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491beed0-a617-40ec-8db1-a92c7030d558 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:57.323818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491beed0-a617-40ec-8db1-a92c7030d558 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:57.325012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491beed0-a617-40ec-8db1-a92c7030d558 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:57.330960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 431/850] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:07:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:57.418087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 420/851] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:57 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:07:57.425549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491beed0-a617-40ec-8db1-a92c7030d558 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:57.522207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17563e-7459-4a94-9894-abf56fd2ee6a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:57.522562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17563e-7459-4a94-9894-abf56fd2ee6a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:57.522744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17563e-7459-4a94-9894-abf56fd2ee6a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:57.524850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 421/852] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:07:57.531917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89a59a-6ac7-4920-ad91-7809027ec0b3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:57.532192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89a59a-6ac7-4920-ad91-7809027ec0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:57.532531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc89a59a-6ac7-4920-ad91-7809027ec0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:57.543242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491beed0-a617-40ec-8db1-a92c7030d558 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:57.553339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cc89a59a-6ac7-4920-ad91-7809027ec0b3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:57.554569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cc89a59a-6ac7-4920-ad91-7809027ec0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:57.587384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412eb3f-200a-475d-a38f-7c08d79040fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:57.588587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412eb3f-200a-475d-a38f-7c08d79040fa None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:57.589017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412eb3f-200a-475d-a38f-7c08d79040fa None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:57.589281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2412eb3f-200a-475d-a38f-7c08d79040fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:57.604389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412eb3f-200a-475d-a38f-7c08d79040fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:57.606308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2412eb3f-200a-475d-a38f-7c08d79040fa None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:57.616678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2412eb3f-200a-475d-a38f-7c08d79040fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:57.629741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:57.630934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:57.632001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:57.632001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:57.632001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:57.633751 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 432/854] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:57.635820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 422/853] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:07:57.639056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412eb3f-200a-475d-a38f-7c08d79040fa None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:57.676288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:57.745902 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2412eb3f-200a-475d-a38f-7c08d79040fa None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:57.782660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375534-e8dc-4879-816c-47b2336e5cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:57.783324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375534-e8dc-4879-816c-47b2336e5cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:57.783610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375534-e8dc-4879-816c-47b2336e5cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:57.783910 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375534-e8dc-4879-816c-47b2336e5cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:57.784602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92375534-e8dc-4879-816c-47b2336e5cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:57.784906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92375534-e8dc-4879-816c-47b2336e5cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:57.786272 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92375534-e8dc-4879-816c-47b2336e5cd6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:57.798887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 433/855] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:57.804407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89a59a-6ac7-4920-ad91-7809027ec0b3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:57.907086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375534-e8dc-4879-816c-47b2336e5cd6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:57.918831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc89a59a-6ac7-4920-ad91-7809027ec0b3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:57.960200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667b0b5-db65-4e60-81e7-1b7cc812288f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:57.961050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667b0b5-db65-4e60-81e7-1b7cc812288f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:57.961381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667b0b5-db65-4e60-81e7-1b7cc812288f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:57.961676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667b0b5-db65-4e60-81e7-1b7cc812288f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:57.970853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667b0b5-db65-4e60-81e7-1b7cc812288f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:57.973980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667b0b5-db65-4e60-81e7-1b7cc812288f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0667b0b5-db65-4e60-81e7-1b7cc812288f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:57.989103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:57.990068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:57.990937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:57.990937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:57.990937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:57.990937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:57.990937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:57.990937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:57.990937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:57.990937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:57.990937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 423/856] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:57.996971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667b0b5-db65-4e60-81e7-1b7cc812288f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:58.029442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:58.057835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375534-e8dc-4879-816c-47b2336e5cd6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:58.062526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:58.098107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156e448-41cf-41b3-8d5d-039fde6b33f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:58.098753 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156e448-41cf-41b3-8d5d-039fde6b33f1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:58.099063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156e448-41cf-41b3-8d5d-039fde6b33f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:58.099442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156e448-41cf-41b3-8d5d-039fde6b33f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:58.111472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156e448-41cf-41b3-8d5d-039fde6b33f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1156e448-41cf-41b3-8d5d-039fde6b33f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:07:58.116766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:58.117740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:58.118610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 434/857] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:07:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:58.126156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156e448-41cf-41b3-8d5d-039fde6b33f1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:58.128930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667b0b5-db65-4e60-81e7-1b7cc812288f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:58.176409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf5d6-3ea3-432d-9acf-95865e20c0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:58.177386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf5d6-3ea3-432d-9acf-95865e20c0bc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:58.177867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf5d6-3ea3-432d-9acf-95865e20c0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:58.178374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bf5d6-3ea3-432d-9acf-95865e20c0bc None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:58.179900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bf5d6-3ea3-432d-9acf-95865e20c0bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:58.180476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bf5d6-3ea3-432d-9acf-95865e20c0bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:58.182983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518bf5d6-3ea3-432d-9acf-95865e20c0bc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:58.202052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 424/858] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:58.209694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf5d6-3ea3-432d-9acf-95865e20c0bc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:58.265177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156e448-41cf-41b3-8d5d-039fde6b33f1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:58.302800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48d155-6b27-4d19-9513-85ecb9a0b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:58.303463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48d155-6b27-4d19-9513-85ecb9a0b308 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:58.303771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48d155-6b27-4d19-9513-85ecb9a0b308 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:58.304748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48d155-6b27-4d19-9513-85ecb9a0b308 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:58.304748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48d155-6b27-4d19-9513-85ecb9a0b308 None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:58.305060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48d155-6b27-4d19-9513-85ecb9a0b308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:58.306518 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48d155-6b27-4d19-9513-85ecb9a0b308 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:58.319817 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 435/859] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:07:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:58.320845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48d155-6b27-4d19-9513-85ecb9a0b308 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:58.324733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:58.350384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bf5d6-3ea3-432d-9acf-95865e20c0bc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:58.365363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:58.397901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3fbab-88c4-47dd-883c-dcaa33721afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:58.398771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3fbab-88c4-47dd-883c-dcaa33721afa None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:58.400055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3fbab-88c4-47dd-883c-dcaa33721afa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:58.400488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3fbab-88c4-47dd-883c-dcaa33721afa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:58.402221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3fbab-88c4-47dd-883c-dcaa33721afa None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=e0bf75c730c946bc9168dd418e7af637) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:58.402687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3fbab-88c4-47dd-883c-dcaa33721afa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:58.404305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3fbab-88c4-47dd-883c-dcaa33721afa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:58.412181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 425/860] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:07:58 2022] GET /identity/v3/role_assignments?user.id=e0bf75c730c946bc9168dd418e7af637&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:07:58.436026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48d155-6b27-4d19-9513-85ecb9a0b308 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:58.476215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff965d4-090a-4d80-ae80-92afeb96727a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:58.477165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff965d4-090a-4d80-ae80-92afeb96727a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:58.477639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff965d4-090a-4d80-ae80-92afeb96727a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:58.478058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff965d4-090a-4d80-ae80-92afeb96727a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:58.490733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff965d4-090a-4d80-ae80-92afeb96727a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:58.492482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff965d4-090a-4d80-ae80-92afeb96727a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6ff965d4-090a-4d80-ae80-92afeb96727a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:58.509187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:58.510372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:58.511427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:58.511427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:58.511427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:58.513214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 436/861] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:07:58.519140 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3fbab-88c4-47dd-883c-dcaa33721afa None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:58.629959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3fbab-88c4-47dd-883c-dcaa33721afa None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:58.667514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da586508-12f1-4432-8861-8b11964ce273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:58.668422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da586508-12f1-4432-8861-8b11964ce273 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:58.668851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da586508-12f1-4432-8861-8b11964ce273 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:58.669271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da586508-12f1-4432-8861-8b11964ce273 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:58.670116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da586508-12f1-4432-8861-8b11964ce273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:58.670662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da586508-12f1-4432-8861-8b11964ce273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:58.672260 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da586508-12f1-4432-8861-8b11964ce273 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:58.685327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 426/862] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:58.690911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff965d4-090a-4d80-ae80-92afeb96727a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:58.751886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:58.801670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff965d4-090a-4d80-ae80-92afeb96727a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:58.838488 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d746563-e124-46ae-a1a0-9f6b6ca30d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:58.839118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d746563-e124-46ae-a1a0-9f6b6ca30d1f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:58.839398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d746563-e124-46ae-a1a0-9f6b6ca30d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:58.839672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d746563-e124-46ae-a1a0-9f6b6ca30d1f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:58.848030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d746563-e124-46ae-a1a0-9f6b6ca30d1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:58.849473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d746563-e124-46ae-a1a0-9f6b6ca30d1f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4d746563-e124-46ae-a1a0-9f6b6ca30d1f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:07:58.861116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:58.862052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:58.862971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 437/863] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:07:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:58.868930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da586508-12f1-4432-8861-8b11964ce273 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:58.905600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:58.978414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da586508-12f1-4432-8861-8b11964ce273 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:59.015548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36ba37-7a4d-42d5-a5bd-9eff32b94d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:59.016187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36ba37-7a4d-42d5-a5bd-9eff32b94d17 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:59.016470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36ba37-7a4d-42d5-a5bd-9eff32b94d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:59.016746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36ba37-7a4d-42d5-a5bd-9eff32b94d17 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:59.017483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36ba37-7a4d-42d5-a5bd-9eff32b94d17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:59.017788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36ba37-7a4d-42d5-a5bd-9eff32b94d17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:59.019215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36ba37-7a4d-42d5-a5bd-9eff32b94d17 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:59.045484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 427/864] 10.222.0.71 () {56 vars in 1149 bytes} [Wed Sep 21 12:07:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:59.048845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d746563-e124-46ae-a1a0-9f6b6ca30d1f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:59.159573 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d746563-e124-46ae-a1a0-9f6b6ca30d1f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:59.175145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:59.196835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d831ae-1dd1-4a11-b6a8-5992179130e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:59.197449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d831ae-1dd1-4a11-b6a8-5992179130e5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:59.197733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d831ae-1dd1-4a11-b6a8-5992179130e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:59.198007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d831ae-1dd1-4a11-b6a8-5992179130e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:59.270170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d831ae-1dd1-4a11-b6a8-5992179130e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:59.271640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d831ae-1dd1-4a11-b6a8-5992179130e5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:59.383527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 438/865] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:07:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:07:59.389809 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36ba37-7a4d-42d5-a5bd-9eff32b94d17 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:59.502336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:59.511671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36ba37-7a4d-42d5-a5bd-9eff32b94d17 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:59.548405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a805f-5bb8-474a-ba11-0c16bce28d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:59.549043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a805f-5bb8-474a-ba11-0c16bce28d98 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:59.549319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a805f-5bb8-474a-ba11-0c16bce28d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:59.549587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a805f-5bb8-474a-ba11-0c16bce28d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:59.553967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd9093-1a53-4acf-a6fd-c79354890c5b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:59.554255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd9093-1a53-4acf-a6fd-c79354890c5b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:59.554534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd9093-1a53-4acf-a6fd-c79354890c5b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:59.556247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 439/866] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:59.560293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a805f-5bb8-474a-ba11-0c16bce28d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:59.561600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476dbec-2c83-43a8-9f31-e4e8099931e8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:59.561817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a805f-5bb8-474a-ba11-0c16bce28d98 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:59.561938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476dbec-2c83-43a8-9f31-e4e8099931e8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:59.562153 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0476dbec-2c83-43a8-9f31-e4e8099931e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e24a805f-5bb8-474a-ba11-0c16bce28d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:07:59.577111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:59.578272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:07:59.579331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:07:59.579331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:07:59.579331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:07:59.580228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0476dbec-2c83-43a8-9f31-e4e8099931e8 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:59.581210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 428/867] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:59.581478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0476dbec-2c83-43a8-9f31-e4e8099931e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:59.589496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a805f-5bb8-474a-ba11-0c16bce28d98 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:59.592308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:59.656534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 440/868] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:07:59.658183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca03a00-6e81-4ba8-863e-e9574df43052 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:59.658474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca03a00-6e81-4ba8-863e-e9574df43052 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:59.658749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca03a00-6e81-4ba8-863e-e9574df43052 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:59.660138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 441/869] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:07:59.666016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028488b0-eef8-41d1-b2e0-6b653d3e7b09 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:59.666366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028488b0-eef8-41d1-b2e0-6b653d3e7b09 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:59.666564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028488b0-eef8-41d1-b2e0-6b653d3e7b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:59.686332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-028488b0-eef8-41d1-b2e0-6b653d3e7b09 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:07:59.687526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-028488b0-eef8-41d1-b2e0-6b653d3e7b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:07:59.731964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a805f-5bb8-474a-ba11-0c16bce28d98 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:59.780338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2f646-94a4-4e5f-975a-515b2ab77c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:59.787776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2f646-94a4-4e5f-975a-515b2ab77c79 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:59.787969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 442/870] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:07:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:07:59.788366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2f646-94a4-4e5f-975a-515b2ab77c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:59.788366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2f646-94a4-4e5f-975a-515b2ab77c79 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:59.788366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2f646-94a4-4e5f-975a-515b2ab77c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:07:59.788366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2f646-94a4-4e5f-975a-515b2ab77c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:59.788366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2f646-94a4-4e5f-975a-515b2ab77c79 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:59.804243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 429/871] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:07:59.805725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028488b0-eef8-41d1-b2e0-6b653d3e7b09 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:59.887167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:59.915448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028488b0-eef8-41d1-b2e0-6b653d3e7b09 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:07:59.926668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2f646-94a4-4e5f-975a-515b2ab77c79 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:07:59.940099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:07:59.960596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bff1dd-17de-4fe1-a8ec-556f35199b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:07:59.961502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bff1dd-17de-4fe1-a8ec-556f35199b60 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:07:59.962509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bff1dd-17de-4fe1-a8ec-556f35199b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:07:59.962796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bff1dd-17de-4fe1-a8ec-556f35199b60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:07:59.971530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bff1dd-17de-4fe1-a8ec-556f35199b60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:07:59.973021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bff1dd-17de-4fe1-a8ec-556f35199b60 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-83bff1dd-17de-4fe1-a8ec-556f35199b60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:07:59.987597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:07:59.988530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:07:59.989412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:07:59.989412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:07:59.989412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:07:59.989412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:07:59.989412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:07:59.989412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:07:59.989412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:07:59.989412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:07:59.991826 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 443/872] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:07:59.995131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bff1dd-17de-4fe1-a8ec-556f35199b60 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:00.041861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2f646-94a4-4e5f-975a-515b2ab77c79 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:00.078196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295e84d-8a3c-4b8f-a317-cc3388e9dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:00.078795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e84d-8a3c-4b8f-a317-cc3388e9dc21 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:00.079072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e84d-8a3c-4b8f-a317-cc3388e9dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:00.079335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295e84d-8a3c-4b8f-a317-cc3388e9dc21 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:00.080137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295e84d-8a3c-4b8f-a317-cc3388e9dc21 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:00.081595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295e84d-8a3c-4b8f-a317-cc3388e9dc21 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:00.099799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 430/873] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:07:59 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:08:00.099799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295e84d-8a3c-4b8f-a317-cc3388e9dc21 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:00.122937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bff1dd-17de-4fe1-a8ec-556f35199b60 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:00.164509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:00.172335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bb534-5dbb-4d8b-9730-146e3f9e4608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:00.173223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bb534-5dbb-4d8b-9730-146e3f9e4608 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:00.173652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bb534-5dbb-4d8b-9730-146e3f9e4608 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:00.174098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bb534-5dbb-4d8b-9730-146e3f9e4608 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:00.175214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bb534-5dbb-4d8b-9730-146e3f9e4608 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:00.175851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bb534-5dbb-4d8b-9730-146e3f9e4608 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:00.177508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bb534-5dbb-4d8b-9730-146e3f9e4608 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:00.183624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:00.198280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 444/874] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:00.199730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bb534-5dbb-4d8b-9730-146e3f9e4608 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:00.234348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295e84d-8a3c-4b8f-a317-cc3388e9dc21 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:00.279579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32f1d9-76fe-413f-a52a-a864ee43daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:00.280572 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32f1d9-76fe-413f-a52a-a864ee43daec None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:00.281079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32f1d9-76fe-413f-a52a-a864ee43daec None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:00.281905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32f1d9-76fe-413f-a52a-a864ee43daec None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:00.283578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32f1d9-76fe-413f-a52a-a864ee43daec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:00.285816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32f1d9-76fe-413f-a52a-a864ee43daec None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:00.307525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 431/875] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:08:00 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:00.323267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bb534-5dbb-4d8b-9730-146e3f9e4608 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:00.375147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016f0c9-ff4c-433b-888c-42441029266f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:00.376227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016f0c9-ff4c-433b-888c-42441029266f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:00.376716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016f0c9-ff4c-433b-888c-42441029266f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:00.377193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016f0c9-ff4c-433b-888c-42441029266f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:00.378196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016f0c9-ff4c-433b-888c-42441029266f None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=376fd55b948f49fda100a7da983f285c) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:00.378734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016f0c9-ff4c-433b-888c-42441029266f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:00.380534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016f0c9-ff4c-433b-888c-42441029266f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:00.388190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 445/876] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:00 2022] GET /identity/v3/role_assignments?user.id=376fd55b948f49fda100a7da983f285c&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:00.756748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7cac-7920-4761-a52e-7d17c85f4de3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:00.757297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7cac-7920-4761-a52e-7d17c85f4de3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:00.757297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7cac-7920-4761-a52e-7d17c85f4de3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:00.759444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 432/877] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:00.765094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119706de-3099-4b5b-acde-b83993bd8f93 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:00.765385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119706de-3099-4b5b-acde-b83993bd8f93 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:00.765668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119706de-3099-4b5b-acde-b83993bd8f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:00.784652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-119706de-3099-4b5b-acde-b83993bd8f93 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:00.785860 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-119706de-3099-4b5b-acde-b83993bd8f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:00.833872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:00.856484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 446/878] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:01.127928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7cac-7920-4761-a52e-7d17c85f4de3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:01.235739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7cac-7920-4761-a52e-7d17c85f4de3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:01.264282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:01.277002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2ff04-3907-4cfe-8442-f220fddf36ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:01.277619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2ff04-3907-4cfe-8442-f220fddf36ab None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:01.277900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2ff04-3907-4cfe-8442-f220fddf36ab None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:01.278172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2ff04-3907-4cfe-8442-f220fddf36ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:01.279151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2ff04-3907-4cfe-8442-f220fddf36ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:01.280698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2ff04-3907-4cfe-8442-f220fddf36ab None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-23c2ff04-3907-4cfe-8442-f220fddf36ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:01.290222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:01.291195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:01.292057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 433/879] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:08:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:01.297435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119706de-3099-4b5b-acde-b83993bd8f93 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:01.404563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119706de-3099-4b5b-acde-b83993bd8f93 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:01.441334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7256a9c-4888-491d-97b4-3777b4539b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:01.442342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7256a9c-4888-491d-97b4-3777b4539b39 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:01.442823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7256a9c-4888-491d-97b4-3777b4539b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:01.443292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7256a9c-4888-491d-97b4-3777b4539b39 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:01.444144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7256a9c-4888-491d-97b4-3777b4539b39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:01.444670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7256a9c-4888-491d-97b4-3777b4539b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:01.446741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7256a9c-4888-491d-97b4-3777b4539b39 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:01.452183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 447/880] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:01.460678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2ff04-3907-4cfe-8442-f220fddf36ab None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:01.573006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2ff04-3907-4cfe-8442-f220fddf36ab None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:01.611837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901682a0-038e-424a-90db-3f34370f13fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:01.612449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901682a0-038e-424a-90db-3f34370f13fa None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:01.612789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901682a0-038e-424a-90db-3f34370f13fa None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:01.613068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901682a0-038e-424a-90db-3f34370f13fa None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:01.613697 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901682a0-038e-424a-90db-3f34370f13fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:01.613997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901682a0-038e-424a-90db-3f34370f13fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:01.615368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901682a0-038e-424a-90db-3f34370f13fa None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:01.621036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 434/881] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:02.428496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da6055-0635-4393-a43b-7db2399dc42a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:02.429019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da6055-0635-4393-a43b-7db2399dc42a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:02.429019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da6055-0635-4393-a43b-7db2399dc42a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:02.431255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 448/882] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:02.436594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd5b3b-2f0f-41a8-aee6-e53afed8fa17 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:02.437451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd5b3b-2f0f-41a8-aee6-e53afed8fa17 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:02.437936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd5b3b-2f0f-41a8-aee6-e53afed8fa17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:02.457243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f3dd5b3b-2f0f-41a8-aee6-e53afed8fa17 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:02.458790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f3dd5b3b-2f0f-41a8-aee6-e53afed8fa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:02.512016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:02.541949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 435/883] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:02.637938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b140944-a2a1-4327-911a-83a1c0cd11c0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:02.638730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b140944-a2a1-4327-911a-83a1c0cd11c0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:02.639249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b140944-a2a1-4327-911a-83a1c0cd11c0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:02.641175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 449/884] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:02.647105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e676a2b-6fcf-421a-b5ce-fdbbaee135c3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:02.647393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e676a2b-6fcf-421a-b5ce-fdbbaee135c3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:02.647666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e676a2b-6fcf-421a-b5ce-fdbbaee135c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:02.666989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8e676a2b-6fcf-421a-b5ce-fdbbaee135c3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:02.668496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e676a2b-6fcf-421a-b5ce-fdbbaee135c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:02.739252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544afa5-6445-45a8-bfbe-6516490e5a96 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:02.739896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544afa5-6445-45a8-bfbe-6516490e5a96 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:02.740394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544afa5-6445-45a8-bfbe-6516490e5a96 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:02.740521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 436/885] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:02.742009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 450/886] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:02.753335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3011e4-0830-4c44-a8a9-e01f7e57c655 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:02.753619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3011e4-0830-4c44-a8a9-e01f7e57c655 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:02.753888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3011e4-0830-4c44-a8a9-e01f7e57c655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:02.778202 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6c3011e4-0830-4c44-a8a9-e01f7e57c655 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:02.779382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6c3011e4-0830-4c44-a8a9-e01f7e57c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:02.814132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544afa5-6445-45a8-bfbe-6516490e5a96 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:02.849073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 437/887] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:02.877126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:02.929946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544afa5-6445-45a8-bfbe-6516490e5a96 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:02.966601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c7f64-1747-4a39-b133-12da1d294a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:02.967243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7f64-1747-4a39-b133-12da1d294a5b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:02.967523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7f64-1747-4a39-b133-12da1d294a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:02.967821 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c7f64-1747-4a39-b133-12da1d294a5b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:02.968411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c7f64-1747-4a39-b133-12da1d294a5b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:02.969780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c7f64-1747-4a39-b133-12da1d294a5b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-092c7f64-1747-4a39-b133-12da1d294a5b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:02.979777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:02.980717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 12:08:02.981576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:02.985845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 451/888] 10.222.0.71 () {56 vars in 1132 bytes} [Wed Sep 21 12:08:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:02.987165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3011e4-0830-4c44-a8a9-e01f7e57c655 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.030375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c7f64-1747-4a39-b133-12da1d294a5b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.034230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:03.092927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3011e4-0830-4c44-a8a9-e01f7e57c655 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.130148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99365e-bb44-41fd-92fa-18f959bdf0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.130797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99365e-bb44-41fd-92fa-18f959bdf0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.131039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99365e-bb44-41fd-92fa-18f959bdf0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.131315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99365e-bb44-41fd-92fa-18f959bdf0d5 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.131993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99365e-bb44-41fd-92fa-18f959bdf0d5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:03.132298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99365e-bb44-41fd-92fa-18f959bdf0d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:03.133874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99365e-bb44-41fd-92fa-18f959bdf0d5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:03.143937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 438/889] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:08:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:03.144825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99365e-bb44-41fd-92fa-18f959bdf0d5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.159275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c7f64-1747-4a39-b133-12da1d294a5b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.169782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:03.196392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce5ed5-90a8-4241-af95-453caed93ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.197191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce5ed5-90a8-4241-af95-453caed93ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.197475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce5ed5-90a8-4241-af95-453caed93ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.197752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce5ed5-90a8-4241-af95-453caed93ed5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.198331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce5ed5-90a8-4241-af95-453caed93ed5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:03.199693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce5ed5-90a8-4241-af95-453caed93ed5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-7cce5ed5-90a8-4241-af95-453caed93ed5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:03.208500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:03.209690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 12:08:03.210791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:03.214934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 452/890] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:03.215741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce5ed5-90a8-4241-af95-453caed93ed5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.261499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99365e-bb44-41fd-92fa-18f959bdf0d5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.270444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:03.303243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e03cb9-dbef-4b7c-99e7-fbc6f2c8f999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.303789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e03cb9-dbef-4b7c-99e7-fbc6f2c8f999 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.304068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e03cb9-dbef-4b7c-99e7-fbc6f2c8f999 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.304340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e03cb9-dbef-4b7c-99e7-fbc6f2c8f999 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.314830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e03cb9-dbef-4b7c-99e7-fbc6f2c8f999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-86e03cb9-dbef-4b7c-99e7-fbc6f2c8f999 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:03.320194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:03.321324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 439/891] 10.222.0.71 () {56 vars in 1120 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:03.322215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e03cb9-dbef-4b7c-99e7-fbc6f2c8f999 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.329485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce5ed5-90a8-4241-af95-453caed93ed5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.366719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b446b-306d-4435-ab3a-0265bca77f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.366975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b446b-306d-4435-ab3a-0265bca77f9f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.367223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b446b-306d-4435-ab3a-0265bca77f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.367417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b446b-306d-4435-ab3a-0265bca77f9f None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.368097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b446b-306d-4435-ab3a-0265bca77f9f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:03.368398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b446b-306d-4435-ab3a-0265bca77f9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:03.369760 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b446b-306d-4435-ab3a-0265bca77f9f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:03.379464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 453/892] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:03.379464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b446b-306d-4435-ab3a-0265bca77f9f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.421735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:03.439239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e03cb9-dbef-4b7c-99e7-fbc6f2c8f999 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.476641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46297f21-6c4c-4d3b-9002-ca80bb12447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.477435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46297f21-6c4c-4d3b-9002-ca80bb12447a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.477715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46297f21-6c4c-4d3b-9002-ca80bb12447a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.477992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46297f21-6c4c-4d3b-9002-ca80bb12447a None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.478619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46297f21-6c4c-4d3b-9002-ca80bb12447a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:03.478922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46297f21-6c4c-4d3b-9002-ca80bb12447a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:03.480343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46297f21-6c4c-4d3b-9002-ca80bb12447a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:03.490038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 440/893] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:03.490038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46297f21-6c4c-4d3b-9002-ca80bb12447a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.497997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b446b-306d-4435-ab3a-0265bca77f9f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.521937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:03.537733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2830355-44e3-4e96-b70c-3ab0f88909b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.538333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2830355-44e3-4e96-b70c-3ab0f88909b2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.538614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2830355-44e3-4e96-b70c-3ab0f88909b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.538889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2830355-44e3-4e96-b70c-3ab0f88909b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.539579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2830355-44e3-4e96-b70c-3ab0f88909b2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:03.539929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2830355-44e3-4e96-b70c-3ab0f88909b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:03.541407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2830355-44e3-4e96-b70c-3ab0f88909b2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:03.549638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 454/894] 10.222.0.71 () {56 vars in 1115 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:03.550459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2830355-44e3-4e96-b70c-3ab0f88909b2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.612084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46297f21-6c4c-4d3b-9002-ca80bb12447a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.656417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a32b6-b5d5-4e0d-a831-394e1fd21cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.657204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a32b6-b5d5-4e0d-a831-394e1fd21cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.657486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a32b6-b5d5-4e0d-a831-394e1fd21cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.658159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a32b6-b5d5-4e0d-a831-394e1fd21cd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.658879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a32b6-b5d5-4e0d-a831-394e1fd21cd0 None admin] RBAC: Adding query filter params (interface=public, service_id=fe8d73f44e0c4f19acc4f0b7b3428fce, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:03.659188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a32b6-b5d5-4e0d-a831-394e1fd21cd0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:03.660594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a32b6-b5d5-4e0d-a831-394e1fd21cd0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:03.671988 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 441/895] 10.222.0.71 () {56 vars in 1251 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/endpoints?service_id=fe8d73f44e0c4f19acc4f0b7b3428fce&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:03.672764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a32b6-b5d5-4e0d-a831-394e1fd21cd0 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.673751 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:03.679208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2830355-44e3-4e96-b70c-3ab0f88909b2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.716650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7831-e58b-4ab4-a1e6-c6abadb729e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.717257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7831-e58b-4ab4-a1e6-c6abadb729e7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.717537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7831-e58b-4ab4-a1e6-c6abadb729e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.717832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da7831-e58b-4ab4-a1e6-c6abadb729e7 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.718456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7831-e58b-4ab4-a1e6-c6abadb729e7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:03.718753 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7831-e58b-4ab4-a1e6-c6abadb729e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:03.720143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da7831-e58b-4ab4-a1e6-c6abadb729e7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:03.725237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 455/896] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:03.728824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7831-e58b-4ab4-a1e6-c6abadb729e7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.791001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:03.799818 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a32b6-b5d5-4e0d-a831-394e1fd21cd0 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.839937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8d7c2-3565-4d6e-9219-6ac92bb3e344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.840567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8d7c2-3565-4d6e-9219-6ac92bb3e344 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.840845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8d7c2-3565-4d6e-9219-6ac92bb3e344 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.841124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d8d7c2-3565-4d6e-9219-6ac92bb3e344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.851159 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da7831-e58b-4ab4-a1e6-c6abadb729e7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:03.852217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8d7c2-3565-4d6e-9219-6ac92bb3e344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:03.853664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d8d7c2-3565-4d6e-9219-6ac92bb3e344 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e9d8d7c2-3565-4d6e-9219-6ac92bb3e344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:03.869409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:03.870338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:03.871215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:03.871215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:03.871215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:03.871215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 442/897] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:03.871215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8d7c2-3565-4d6e-9219-6ac92bb3e344 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.890435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90825c4-2d94-4015-a5b5-8661cd9dcd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:03.891333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90825c4-2d94-4015-a5b5-8661cd9dcd67 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:03.891789 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90825c4-2d94-4015-a5b5-8661cd9dcd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:03.892077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90825c4-2d94-4015-a5b5-8661cd9dcd67 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:03.892679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90825c4-2d94-4015-a5b5-8661cd9dcd67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:03.894050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90825c4-2d94-4015-a5b5-8661cd9dcd67 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:03.900184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 456/898] 10.222.0.71 () {56 vars in 1087 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:08:03.900841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90825c4-2d94-4015-a5b5-8661cd9dcd67 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:03.935660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:04.008051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d8d7c2-3565-4d6e-9219-6ac92bb3e344 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:04.017930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90825c4-2d94-4015-a5b5-8661cd9dcd67 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:04.045603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3d70e-8247-4d8c-9264-fec91dde5cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:04.046706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3d70e-8247-4d8c-9264-fec91dde5cff None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:04.047052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3d70e-8247-4d8c-9264-fec91dde5cff None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:04.047335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3d70e-8247-4d8c-9264-fec91dde5cff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:04.048108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3d70e-8247-4d8c-9264-fec91dde5cff None admin] RBAC: Adding query filter params (interface=public, service_id=96c41fa9f5284bb6b6da3efe316a7dbe, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:04.048418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3d70e-8247-4d8c-9264-fec91dde5cff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:04.049833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3d70e-8247-4d8c-9264-fec91dde5cff None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:04.056145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4404f9-b7fc-4cd8-adeb-e19e1f401213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:04.056639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 443/899] 10.222.0.71 () {56 vars in 1251 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/endpoints?service_id=96c41fa9f5284bb6b6da3efe316a7dbe&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:04.057935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2e53-81f3-404d-bd1f-6c68846de5d5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:04.058223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2e53-81f3-404d-bd1f-6c68846de5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:04.058494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2e53-81f3-404d-bd1f-6c68846de5d5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:04.059923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 444/900] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:04.061507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4404f9-b7fc-4cd8-adeb-e19e1f401213 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:04.061801 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4404f9-b7fc-4cd8-adeb-e19e1f401213 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:04.062223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4404f9-b7fc-4cd8-adeb-e19e1f401213 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:04.062919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4404f9-b7fc-4cd8-adeb-e19e1f401213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:04.063227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4404f9-b7fc-4cd8-adeb-e19e1f401213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:04.064733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4404f9-b7fc-4cd8-adeb-e19e1f401213 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:04.064855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2e53-81f3-404d-bd1f-6c68846de5d5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:04.083087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 457/901] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:04.083320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:04.083413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac558b-755b-464b-95ea-99b108754965 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:04.083413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac558b-755b-464b-95ea-99b108754965 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:04.083413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac558b-755b-464b-95ea-99b108754965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:04.101336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ddac558b-755b-464b-95ea-99b108754965 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:04.103020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ddac558b-755b-464b-95ea-99b108754965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:04.178387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 458/902] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:04.179174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac558b-755b-464b-95ea-99b108754965 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:04.192996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2e53-81f3-404d-bd1f-6c68846de5d5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:04.200940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:04.240358 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e7e97-933d-4b5c-b530-d9e259313a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:04.241600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e7e97-933d-4b5c-b530-d9e259313a61 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:04.242480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e7e97-933d-4b5c-b530-d9e259313a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:04.243552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e7e97-933d-4b5c-b530-d9e259313a61 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:04.244384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e7e97-933d-4b5c-b530-d9e259313a61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:04.246545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e7e97-933d-4b5c-b530-d9e259313a61 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:04.254678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 445/903] 10.222.0.71 () {56 vars in 1087 bytes} [Wed Sep 21 12:08:04 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:08:04.307612 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac558b-755b-464b-95ea-99b108754965 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:04.348594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a1f03-34fd-437d-b7e7-72768ab3cc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:04.349502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1f03-34fd-437d-b7e7-72768ab3cc29 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:04.349954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1f03-34fd-437d-b7e7-72768ab3cc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:04.350369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1f03-34fd-437d-b7e7-72768ab3cc29 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:04.359119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a1f03-34fd-437d-b7e7-72768ab3cc29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:04.360896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a1f03-34fd-437d-b7e7-72768ab3cc29 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-494a1f03-34fd-437d-b7e7-72768ab3cc29 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:08:04.374538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:04.375733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:08:04.376877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:04.379145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 459/904] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:08:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:04.385746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e7e97-933d-4b5c-b530-d9e259313a61 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:04.467804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a1f03-34fd-437d-b7e7-72768ab3cc29 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:04.474887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:04.507965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e7e97-933d-4b5c-b530-d9e259313a61 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:04.551273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70184ee7-ebe5-48b1-b3b5-9bd1d7a8f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:04.551921 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70184ee7-ebe5-48b1-b3b5-9bd1d7a8f178 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:04.552205 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70184ee7-ebe5-48b1-b3b5-9bd1d7a8f178 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:04.552533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70184ee7-ebe5-48b1-b3b5-9bd1d7a8f178 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:04.553271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70184ee7-ebe5-48b1-b3b5-9bd1d7a8f178 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:04.553624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70184ee7-ebe5-48b1-b3b5-9bd1d7a8f178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:04.555070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70184ee7-ebe5-48b1-b3b5-9bd1d7a8f178 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:04.559429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:04.577761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 446/905] 10.222.0.71 () {56 vars in 1149 bytes} [Wed Sep 21 12:08:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:04.583146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70184ee7-ebe5-48b1-b3b5-9bd1d7a8f178 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:04.588874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a1f03-34fd-437d-b7e7-72768ab3cc29 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:04.632553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c634b0-5418-45b9-a1e7-e23a29fcbbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:04.634070 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c634b0-5418-45b9-a1e7-e23a29fcbbb8 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:04.634356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c634b0-5418-45b9-a1e7-e23a29fcbbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:04.634629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c634b0-5418-45b9-a1e7-e23a29fcbbb8 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:04.635301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c634b0-5418-45b9-a1e7-e23a29fcbbb8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:04.636938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c634b0-5418-45b9-a1e7-e23a29fcbbb8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:04.656284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 460/906] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:08:04 2022] POST /identity/v3/services => generated 241 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:08:04.703532 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70184ee7-ebe5-48b1-b3b5-9bd1d7a8f178 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:04.740969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:04.754466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291d0aa-dd30-4338-9d14-a8e081cdd561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:04.755388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291d0aa-dd30-4338-9d14-a8e081cdd561 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:04.755859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291d0aa-dd30-4338-9d14-a8e081cdd561 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:04.756271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291d0aa-dd30-4338-9d14-a8e081cdd561 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:04.835690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291d0aa-dd30-4338-9d14-a8e081cdd561 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:04.837232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5291d0aa-dd30-4338-9d14-a8e081cdd561 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:04.947423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 447/907] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:04 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:08:04.952879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c634b0-5418-45b9-a1e7-e23a29fcbbb8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:05.074895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c634b0-5418-45b9-a1e7-e23a29fcbbb8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:05.111013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d055af-7d54-447d-92b1-8f05761bb6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:05.111654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d055af-7d54-447d-92b1-8f05761bb6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:05.111963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d055af-7d54-447d-92b1-8f05761bb6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:05.112231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d055af-7d54-447d-92b1-8f05761bb6c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:05.114998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:05.123994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d055af-7d54-447d-92b1-8f05761bb6c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:05.125452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d055af-7d54-447d-92b1-8f05761bb6c1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-18d055af-7d54-447d-92b1-8f05761bb6c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:05.140744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:05.141959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:05.142842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:05.142842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:05.142842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:05.142842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 461/908] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:05.147871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291d0aa-dd30-4338-9d14-a8e081cdd561 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:05.177131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:05.255315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291d0aa-dd30-4338-9d14-a8e081cdd561 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:05.292133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdec83-e2e6-4453-8201-248c9e225686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:05.293039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdec83-e2e6-4453-8201-248c9e225686 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:05.293677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdec83-e2e6-4453-8201-248c9e225686 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:05.294088 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdec83-e2e6-4453-8201-248c9e225686 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:05.294877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdec83-e2e6-4453-8201-248c9e225686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:05.295317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdec83-e2e6-4453-8201-248c9e225686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:05.296850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bdec83-e2e6-4453-8201-248c9e225686 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:05.310189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 448/909] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:05.315982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d055af-7d54-447d-92b1-8f05761bb6c1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:05.427961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d055af-7d54-447d-92b1-8f05761bb6c1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:05.468034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18110e25-58f7-47a4-b44b-e9e872cf791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:05.468668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18110e25-58f7-47a4-b44b-e9e872cf791e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:05.468945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18110e25-58f7-47a4-b44b-e9e872cf791e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:05.469215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18110e25-58f7-47a4-b44b-e9e872cf791e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:05.476304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18110e25-58f7-47a4-b44b-e9e872cf791e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:05.477781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18110e25-58f7-47a4-b44b-e9e872cf791e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-18110e25-58f7-47a4-b44b-e9e872cf791e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:08:05.489541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:05.490632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:05.491743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:08:05.491743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:08:05.491743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:08:05.491743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:08:05.491743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:08:05.491743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:08:05.491743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:05.491743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:05.491743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 462/910] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:05.499539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdec83-e2e6-4453-8201-248c9e225686 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:05.591665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:05.604522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdec83-e2e6-4453-8201-248c9e225686 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:05.643897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f89b0-b665-4a39-92dd-8405316e7d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:05.644510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f89b0-b665-4a39-92dd-8405316e7d0a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:05.644784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f89b0-b665-4a39-92dd-8405316e7d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:05.645049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f89b0-b665-4a39-92dd-8405316e7d0a None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:05.645746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f89b0-b665-4a39-92dd-8405316e7d0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:05.646043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f89b0-b665-4a39-92dd-8405316e7d0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:05.647685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f89b0-b665-4a39-92dd-8405316e7d0a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:05.662753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 449/911] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:05.671492 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18110e25-58f7-47a4-b44b-e9e872cf791e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:05.733077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:05.787596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18110e25-58f7-47a4-b44b-e9e872cf791e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:05.822403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300012a-2734-4a0a-bfcc-5df78d374c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:05.823014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300012a-2734-4a0a-bfcc-5df78d374c40 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:05.823290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300012a-2734-4a0a-bfcc-5df78d374c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:05.823559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300012a-2734-4a0a-bfcc-5df78d374c40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:05.824318 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300012a-2734-4a0a-bfcc-5df78d374c40 None admin] RBAC: Adding query filter params (role.id=d1f99f1b165f4a919c871bd6722ea445, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=376fd55b948f49fda100a7da983f285c) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:05.824624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300012a-2734-4a0a-bfcc-5df78d374c40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:05.826069 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300012a-2734-4a0a-bfcc-5df78d374c40 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:05.831947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 463/912] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:05 2022] GET /identity/v3/role_assignments?user.id=376fd55b948f49fda100a7da983f285c&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=d1f99f1b165f4a919c871bd6722ea445 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:06.238702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8f6ec-904a-488b-8161-38d737bd63bb None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:06.239059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8f6ec-904a-488b-8161-38d737bd63bb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:06.239292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8f6ec-904a-488b-8161-38d737bd63bb None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:06.240950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 450/913] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:06.246301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1c43a-96d4-4ed0-bb35-87286d82b493 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:06.246594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1c43a-96d4-4ed0-bb35-87286d82b493 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:06.246871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1c43a-96d4-4ed0-bb35-87286d82b493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:06.265491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c6d1c43a-96d4-4ed0-bb35-87286d82b493 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:06.266684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c6d1c43a-96d4-4ed0-bb35-87286d82b493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:06.336893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 464/914] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:06.568613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708580f5-104e-4b7e-a1bd-d95d7d0c0684 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:06.569130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708580f5-104e-4b7e-a1bd-d95d7d0c0684 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:06.569130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708580f5-104e-4b7e-a1bd-d95d7d0c0684 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:06.571317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 451/915] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:06.576919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ef159-c003-48bb-a762-cdeea218f1fe None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:06.577211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ef159-c003-48bb-a762-cdeea218f1fe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:06.577489 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ef159-c003-48bb-a762-cdeea218f1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:06.596407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-909ef159-c003-48bb-a762-cdeea218f1fe None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:06.597609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-909ef159-c003-48bb-a762-cdeea218f1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:06.605959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708580f5-104e-4b7e-a1bd-d95d7d0c0684 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:06.639168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:06.675187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 465/916] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:06.719451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708580f5-104e-4b7e-a1bd-d95d7d0c0684 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:06.754323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b162c7a-a9ee-4746-9af9-3f41610936ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:06.754892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b162c7a-a9ee-4746-9af9-3f41610936ac None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:06.755175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b162c7a-a9ee-4746-9af9-3f41610936ac None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:06.755442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b162c7a-a9ee-4746-9af9-3f41610936ac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:06.756068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b162c7a-a9ee-4746-9af9-3f41610936ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:06.757610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b162c7a-a9ee-4746-9af9-3f41610936ac None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-5b162c7a-a9ee-4746-9af9-3f41610936ac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:06.769164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:06.770146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 12:08:06.771012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:06.775954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 452/917] 10.222.0.71 () {56 vars in 1132 bytes} [Wed Sep 21 12:08:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:06.777583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ef159-c003-48bb-a762-cdeea218f1fe None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:06.882689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ef159-c003-48bb-a762-cdeea218f1fe None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:06.927258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f04007-636e-4953-b287-5d1e7427d37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:06.927967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f04007-636e-4953-b287-5d1e7427d37b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:06.928246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f04007-636e-4953-b287-5d1e7427d37b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:06.928521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f04007-636e-4953-b287-5d1e7427d37b None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:06.929189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f04007-636e-4953-b287-5d1e7427d37b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:06.929493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f04007-636e-4953-b287-5d1e7427d37b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:06.931417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f04007-636e-4953-b287-5d1e7427d37b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:06.941231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 466/918] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:08:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:06.942610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b162c7a-a9ee-4746-9af9-3f41610936ac None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:06.958578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:06.978072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f04007-636e-4953-b287-5d1e7427d37b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:07.019872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:07.057755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b162c7a-a9ee-4746-9af9-3f41610936ac None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:07.093223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10503af-8c60-4313-ab88-b24bd77e1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:07.093901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10503af-8c60-4313-ab88-b24bd77e1da3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.094113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10503af-8c60-4313-ab88-b24bd77e1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.094386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10503af-8c60-4313-ab88-b24bd77e1da3 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.095016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10503af-8c60-4313-ab88-b24bd77e1da3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:07.095311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10503af-8c60-4313-ab88-b24bd77e1da3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:07.096771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10503af-8c60-4313-ab88-b24bd77e1da3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:07.103443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 453/919] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:08:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:07.104165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f04007-636e-4953-b287-5d1e7427d37b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:07.109249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef51fe-8976-4c82-86bf-8dd7a1fda5b6 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.109794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef51fe-8976-4c82-86bf-8dd7a1fda5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.110311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef51fe-8976-4c82-86bf-8dd7a1fda5b6 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.112092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 454/920] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:07.113520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef51fe-8976-4c82-86bf-8dd7a1fda5b6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:07.142842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644504a-a17d-41b3-a225-f5515f62dc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:07.143479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644504a-a17d-41b3-a225-f5515f62dc73 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.143777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644504a-a17d-41b3-a225-f5515f62dc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.144053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644504a-a17d-41b3-a225-f5515f62dc73 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.144642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644504a-a17d-41b3-a225-f5515f62dc73 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:07.146021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644504a-a17d-41b3-a225-f5515f62dc73 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-d644504a-a17d-41b3-a225-f5515f62dc73 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:07.155189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:07.156237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:07.157075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 467/921] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:08:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:07.158194 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab20d-4687-4872-87ce-e9e7d87c355f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.158194 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab20d-4687-4872-87ce-e9e7d87c355f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.158479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab20d-4687-4872-87ce-e9e7d87c355f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.177821 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4ebab20d-4687-4872-87ce-e9e7d87c355f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:07.179028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4ebab20d-4687-4872-87ce-e9e7d87c355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:07.205444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:07.234251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef51fe-8976-4c82-86bf-8dd7a1fda5b6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:07.250295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 468/922] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:07.251437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebab20d-4687-4872-87ce-e9e7d87c355f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:07.278654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af236c-d685-473c-81a7-2d29cfca3cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:07.279272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af236c-d685-473c-81a7-2d29cfca3cb3 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.279554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af236c-d685-473c-81a7-2d29cfca3cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.279856 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af236c-d685-473c-81a7-2d29cfca3cb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.280663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af236c-d685-473c-81a7-2d29cfca3cb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:07.282149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af236c-d685-473c-81a7-2d29cfca3cb3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:07.300371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:07.315965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 455/923] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:08:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:07.397494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebab20d-4687-4872-87ce-e9e7d87c355f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:07.443199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ea8ea-f70a-4527-8d37-0d93569e85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:07.444257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ea8ea-f70a-4527-8d37-0d93569e85cb None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.444742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ea8ea-f70a-4527-8d37-0d93569e85cb None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.445218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ea8ea-f70a-4527-8d37-0d93569e85cb None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.446908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ea8ea-f70a-4527-8d37-0d93569e85cb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:07.447430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ea8ea-f70a-4527-8d37-0d93569e85cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:07.449078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ea8ea-f70a-4527-8d37-0d93569e85cb None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:07.455881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 469/924] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:08:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:07.463045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af236c-d685-473c-81a7-2d29cfca3cb3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:07.534393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ea8ea-f70a-4527-8d37-0d93569e85cb None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:07.589519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af236c-d685-473c-81a7-2d29cfca3cb3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:07.641240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:07.649131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287879b-16fc-4ab2-963b-f9ebecba3168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:07.650612 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:07.650804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287879b-16fc-4ab2-963b-f9ebecba3168 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.651935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287879b-16fc-4ab2-963b-f9ebecba3168 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.652218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e287879b-16fc-4ab2-963b-f9ebecba3168 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.653039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287879b-16fc-4ab2-963b-f9ebecba3168 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:07.653392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287879b-16fc-4ab2-963b-f9ebecba3168 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:07.654871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e287879b-16fc-4ab2-963b-f9ebecba3168 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:07.662627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ea8ea-f70a-4527-8d37-0d93569e85cb None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:07.666000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 456/925] 10.222.0.71 () {56 vars in 1114 bytes} [Wed Sep 21 12:08:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:07.667086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287879b-16fc-4ab2-963b-f9ebecba3168 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:07.708562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0103e-b301-44af-9f65-3625bddaf8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:07.709233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0103e-b301-44af-9f65-3625bddaf8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.709511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0103e-b301-44af-9f65-3625bddaf8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.709785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0103e-b301-44af-9f65-3625bddaf8f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.710453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0103e-b301-44af-9f65-3625bddaf8f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:07.713008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0103e-b301-44af-9f65-3625bddaf8f8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-72b0103e-b301-44af-9f65-3625bddaf8f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:07.725003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:07.726009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:07.726833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:07.728719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 470/926] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:08:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:07.737388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0103e-b301-44af-9f65-3625bddaf8f8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:07.789513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e287879b-16fc-4ab2-963b-f9ebecba3168 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:07.834949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7a5a0-897f-4df4-b4ee-84649ce49530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:07.836103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a5a0-897f-4df4-b4ee-84649ce49530 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.836712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a5a0-897f-4df4-b4ee-84649ce49530 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.837187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7a5a0-897f-4df4-b4ee-84649ce49530 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.838108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7a5a0-897f-4df4-b4ee-84649ce49530 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:07.839680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7a5a0-897f-4df4-b4ee-84649ce49530 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:07.872010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 457/927] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:08:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:07.876328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0103e-b301-44af-9f65-3625bddaf8f8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:07.925156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af05c04-2d47-4b9d-a370-9fb04f297ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:07.925686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af05c04-2d47-4b9d-a370-9fb04f297ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:07.925962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af05c04-2d47-4b9d-a370-9fb04f297ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:07.926232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af05c04-2d47-4b9d-a370-9fb04f297ae6 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:07.926865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af05c04-2d47-4b9d-a370-9fb04f297ae6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:07.927166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af05c04-2d47-4b9d-a370-9fb04f297ae6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:07.928591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af05c04-2d47-4b9d-a370-9fb04f297ae6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:07.934689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 471/928] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:07.940861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7a5a0-897f-4df4-b4ee-84649ce49530 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:08.020817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:08.076071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7a5a0-897f-4df4-b4ee-84649ce49530 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:08.085428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2802ae2-fa19-4d08-8c20-db5612731c58 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:08.085840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2802ae2-fa19-4d08-8c20-db5612731c58 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:08.086348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2802ae2-fa19-4d08-8c20-db5612731c58 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:08.089951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 472/929] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:08.097566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e67d-625e-43b6-99df-598d6af3d3eb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:08.098593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e67d-625e-43b6-99df-598d6af3d3eb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:08.099059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404e67d-625e-43b6-99df-598d6af3d3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:08.120885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c51a47-0db1-4a61-bdeb-0ddf17ace970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:08.121503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c51a47-0db1-4a61-bdeb-0ddf17ace970 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:08.121765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c51a47-0db1-4a61-bdeb-0ddf17ace970 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:08.122027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c51a47-0db1-4a61-bdeb-0ddf17ace970 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:08.122654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c51a47-0db1-4a61-bdeb-0ddf17ace970 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:08.122947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c51a47-0db1-4a61-bdeb-0ddf17ace970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:08.124336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c51a47-0db1-4a61-bdeb-0ddf17ace970 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:08.129786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 458/930] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:08.140568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c51a47-0db1-4a61-bdeb-0ddf17ace970 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:08.141233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0404e67d-625e-43b6-99df-598d6af3d3eb None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:08.143261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0404e67d-625e-43b6-99df-598d6af3d3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:08.161225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:08.255888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c51a47-0db1-4a61-bdeb-0ddf17ace970 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:08.266078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 473/931] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:08.280468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:08.310925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9653f-bc7c-424b-9f3b-e779512cd127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:08.311931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9653f-bc7c-424b-9f3b-e779512cd127 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:08.312413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9653f-bc7c-424b-9f3b-e779512cd127 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:08.312880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9653f-bc7c-424b-9f3b-e779512cd127 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:08.314135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9653f-bc7c-424b-9f3b-e779512cd127 None admin] RBAC: Adding query filter params (interface=public, service_id=b4f3e8c93dae4709bcd945e068f1f207, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:08.314676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9653f-bc7c-424b-9f3b-e779512cd127 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:08.317113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9653f-bc7c-424b-9f3b-e779512cd127 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:08.326252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 459/932] 10.222.0.71 () {56 vars in 1251 bytes} [Wed Sep 21 12:08:08 2022] GET /identity/v3/endpoints?service_id=b4f3e8c93dae4709bcd945e068f1f207&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:08.334989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404e67d-625e-43b6-99df-598d6af3d3eb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:08.445157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404e67d-625e-43b6-99df-598d6af3d3eb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:08.485765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691f2fe-02b7-417c-b9f9-3c5b51c4d21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:08.486378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691f2fe-02b7-417c-b9f9-3c5b51c4d21e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:08.486656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691f2fe-02b7-417c-b9f9-3c5b51c4d21e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:08.486930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691f2fe-02b7-417c-b9f9-3c5b51c4d21e None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:08.487538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691f2fe-02b7-417c-b9f9-3c5b51c4d21e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:08.489019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691f2fe-02b7-417c-b9f9-3c5b51c4d21e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:08.495741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 474/933] 10.222.0.71 () {56 vars in 1087 bytes} [Wed Sep 21 12:08:08 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:08:08.567309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9653f-bc7c-424b-9f3b-e779512cd127 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:08.682176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9653f-bc7c-424b-9f3b-e779512cd127 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:08.721177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d9202-566e-45a5-bf69-cda311826ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:08.721821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d9202-566e-45a5-bf69-cda311826ffb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:08.722101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d9202-566e-45a5-bf69-cda311826ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:08.722372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d9202-566e-45a5-bf69-cda311826ffb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:08.733449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d9202-566e-45a5-bf69-cda311826ffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-035d9202-566e-45a5-bf69-cda311826ffb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:08.738851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:08.739811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:08.740638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 460/934] 10.222.0.71 () {56 vars in 1120 bytes} [Wed Sep 21 12:08:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:08.749947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691f2fe-02b7-417c-b9f9-3c5b51c4d21e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:08.762551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:08.858989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691f2fe-02b7-417c-b9f9-3c5b51c4d21e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:08.897854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681d1c2-e8cb-4130-811a-6cdb6903d6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:08.898461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681d1c2-e8cb-4130-811a-6cdb6903d6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:08.898772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681d1c2-e8cb-4130-811a-6cdb6903d6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:08.899618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681d1c2-e8cb-4130-811a-6cdb6903d6b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:08.900326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681d1c2-e8cb-4130-811a-6cdb6903d6b2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:08.900627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681d1c2-e8cb-4130-811a-6cdb6903d6b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:08.901995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681d1c2-e8cb-4130-811a-6cdb6903d6b2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:08.911160 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 475/935] 10.222.0.71 () {56 vars in 1115 bytes} [Wed Sep 21 12:08:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:08.916839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d9202-566e-45a5-bf69-cda311826ffb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:08.964693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:09.030501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d9202-566e-45a5-bf69-cda311826ffb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:09.067236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5174e-1c6b-4c07-8981-f7681ed01c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:09.068291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5174e-1c6b-4c07-8981-f7681ed01c55 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:09.068829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5174e-1c6b-4c07-8981-f7681ed01c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:09.069302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5174e-1c6b-4c07-8981-f7681ed01c55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:09.080917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5174e-1c6b-4c07-8981-f7681ed01c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:09.082688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5174e-1c6b-4c07-8981-f7681ed01c55 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-32c5174e-1c6b-4c07-8981-f7681ed01c55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:09.098974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:09.099981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:09.100866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:09.100866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:09.100866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:09.103508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 461/936] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:09.109127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681d1c2-e8cb-4130-811a-6cdb6903d6b2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:09.182388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:09.217344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681d1c2-e8cb-4130-811a-6cdb6903d6b2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:09.253809 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841651b-7de5-4cca-b152-2066cab4d9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:09.254779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841651b-7de5-4cca-b152-2066cab4d9af None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:09.255287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841651b-7de5-4cca-b152-2066cab4d9af None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:09.255798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841651b-7de5-4cca-b152-2066cab4d9af None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:09.256638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841651b-7de5-4cca-b152-2066cab4d9af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:09.257139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841651b-7de5-4cca-b152-2066cab4d9af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:09.258749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841651b-7de5-4cca-b152-2066cab4d9af None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:09.272875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 476/937] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:09.281101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5174e-1c6b-4c07-8981-f7681ed01c55 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:09.385849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:09.396146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5174e-1c6b-4c07-8981-f7681ed01c55 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:09.436588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cb7e3-7fd3-47dd-8eaa-b23188bdae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:09.437220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cb7e3-7fd3-47dd-8eaa-b23188bdae6c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:09.437505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cb7e3-7fd3-47dd-8eaa-b23188bdae6c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:09.437839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cb7e3-7fd3-47dd-8eaa-b23188bdae6c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:09.447107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cb7e3-7fd3-47dd-8eaa-b23188bdae6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:09.448689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cb7e3-7fd3-47dd-8eaa-b23188bdae6c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6c5cb7e3-7fd3-47dd-8eaa-b23188bdae6c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:08:09.460118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:09.461027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:09.462552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 462/938] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:08:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:09.469781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841651b-7de5-4cca-b152-2066cab4d9af None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:09.578722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841651b-7de5-4cca-b152-2066cab4d9af None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:09.608540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:09.614748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795143c1-7ffc-4a7a-93c5-c40d533c75de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:09.615391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795143c1-7ffc-4a7a-93c5-c40d533c75de None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:09.616069 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795143c1-7ffc-4a7a-93c5-c40d533c75de None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:09.616589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795143c1-7ffc-4a7a-93c5-c40d533c75de None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:09.617689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795143c1-7ffc-4a7a-93c5-c40d533c75de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:09.618271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795143c1-7ffc-4a7a-93c5-c40d533c75de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:09.620090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795143c1-7ffc-4a7a-93c5-c40d533c75de None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:09.643402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 477/939] 10.222.0.71 () {56 vars in 1149 bytes} [Wed Sep 21 12:08:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:09.649859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cb7e3-7fd3-47dd-8eaa-b23188bdae6c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:09.766613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cb7e3-7fd3-47dd-8eaa-b23188bdae6c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:09.781346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ca5ab-f644-4c46-92dc-086f861c1558 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:09.781850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ca5ab-f644-4c46-92dc-086f861c1558 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:09.781938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ca5ab-f644-4c46-92dc-086f861c1558 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:09.784194 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 478/940] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:09.789813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d599de-2df5-4ab2-baac-1173087b22c5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:09.790101 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d599de-2df5-4ab2-baac-1173087b22c5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:09.790378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d599de-2df5-4ab2-baac-1173087b22c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:09.807534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d8acd-d0bd-48e2-868f-75bffc012362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:09.808215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d8acd-d0bd-48e2-868f-75bffc012362 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:09.808503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d8acd-d0bd-48e2-868f-75bffc012362 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:09.808774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d8acd-d0bd-48e2-868f-75bffc012362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:09.813491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-01d599de-2df5-4ab2-baac-1173087b22c5 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:09.814953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-01d599de-2df5-4ab2-baac-1173087b22c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:09.820163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d8acd-d0bd-48e2-868f-75bffc012362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:09.822304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d8acd-d0bd-48e2-868f-75bffc012362 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-476d8acd-d0bd-48e2-868f-75bffc012362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:09.840688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:09.841934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:09.842995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:09.842995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:09.842995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:09.842995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 463/941] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:09.848145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d8acd-d0bd-48e2-868f-75bffc012362 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:09.853890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:09.907699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 479/942] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:08:09.973852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d8acd-d0bd-48e2-868f-75bffc012362 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:10.009246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb76c-2f15-4cf7-9187-64f9b968dfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:10.010011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb76c-2f15-4cf7-9187-64f9b968dfcb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.010354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb76c-2f15-4cf7-9187-64f9b968dfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.010698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adb76c-2f15-4cf7-9187-64f9b968dfcb None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.011408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adb76c-2f15-4cf7-9187-64f9b968dfcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:10.011790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adb76c-2f15-4cf7-9187-64f9b968dfcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:10.013291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adb76c-2f15-4cf7-9187-64f9b968dfcb None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:10.026070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 464/943] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:10.031435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d599de-2df5-4ab2-baac-1173087b22c5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:10.139659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d599de-2df5-4ab2-baac-1173087b22c5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:10.157530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:10.182748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd1f08-64e4-4368-9ae2-c25e1a17a451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:10.183806 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1f08-64e4-4368-9ae2-c25e1a17a451 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.184291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1f08-64e4-4368-9ae2-c25e1a17a451 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.184776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd1f08-64e4-4368-9ae2-c25e1a17a451 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.190146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb76c-2f15-4cf7-9187-64f9b968dfcb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:10.193524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd1f08-64e4-4368-9ae2-c25e1a17a451 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:10.195402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd1f08-64e4-4368-9ae2-c25e1a17a451 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e7cd1f08-64e4-4368-9ae2-c25e1a17a451 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:08:10.209541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:10.210487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:10.212560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:08:10.212560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:08:10.212560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:08:10.212560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:08:10.212560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:08:10.212560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:08:10.212560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:10.212560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:10.212560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 480/944] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:10.217228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd1f08-64e4-4368-9ae2-c25e1a17a451 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:10.262941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:10.309186 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adb76c-2f15-4cf7-9187-64f9b968dfcb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:10.330238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd1f08-64e4-4368-9ae2-c25e1a17a451 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:10.347384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f88384-3d03-47c3-b17e-27572a86b954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:10.348035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f88384-3d03-47c3-b17e-27572a86b954 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.348480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f88384-3d03-47c3-b17e-27572a86b954 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.348751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f88384-3d03-47c3-b17e-27572a86b954 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.349638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f88384-3d03-47c3-b17e-27572a86b954 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:10.351009 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f88384-3d03-47c3-b17e-27572a86b954 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-95f88384-3d03-47c3-b17e-27572a86b954 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:10.364407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:10.367962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:10.368807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 465/945] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:10.369618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea533d8c-ae7c-4ba7-ae30-dfd07abadfe7 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.369618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea533d8c-ae7c-4ba7-ae30-dfd07abadfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.369618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea533d8c-ae7c-4ba7-ae30-dfd07abadfe7 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.369618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 466/946] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:10.371316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea533d8c-ae7c-4ba7-ae30-dfd07abadfe7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:10.376638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e072a-a45a-4419-a40a-5bc9becb06ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:10.378086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e072a-a45a-4419-a40a-5bc9becb06ba None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.378385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e072a-a45a-4419-a40a-5bc9becb06ba None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.378663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e072a-a45a-4419-a40a-5bc9becb06ba None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.379367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e072a-a45a-4419-a40a-5bc9becb06ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:10.379677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e072a-a45a-4419-a40a-5bc9becb06ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:10.381140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e072a-a45a-4419-a40a-5bc9becb06ba None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:10.395454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 481/947] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:08:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:10.401246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac80667-03d0-4bfc-a1e8-479efa424546 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.401532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac80667-03d0-4bfc-a1e8-479efa424546 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.401829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac80667-03d0-4bfc-a1e8-479efa424546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.421069 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ac80667-03d0-4bfc-a1e8-479efa424546 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:10.422266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ac80667-03d0-4bfc-a1e8-479efa424546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:10.455508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:10.484732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea533d8c-ae7c-4ba7-ae30-dfd07abadfe7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:10.498765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 482/948] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:10.509843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac80667-03d0-4bfc-a1e8-479efa424546 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:10.519075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:10.533597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ce41c-cc24-4ce6-b85b-4204b50aa928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:10.534218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ce41c-cc24-4ce6-b85b-4204b50aa928 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.534504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ce41c-cc24-4ce6-b85b-4204b50aa928 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.534785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ce41c-cc24-4ce6-b85b-4204b50aa928 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.535709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ce41c-cc24-4ce6-b85b-4204b50aa928 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:10.536058 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ce41c-cc24-4ce6-b85b-4204b50aa928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:10.537904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ce41c-cc24-4ce6-b85b-4204b50aa928 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:10.544742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 467/949] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:08:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:10.549882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ce41c-cc24-4ce6-b85b-4204b50aa928 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:10.631569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac80667-03d0-4bfc-a1e8-479efa424546 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:10.670392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448306-326e-4889-89e8-3bf06f4b65c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:10.671505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448306-326e-4889-89e8-3bf06f4b65c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.672036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448306-326e-4889-89e8-3bf06f4b65c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.672524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24448306-326e-4889-89e8-3bf06f4b65c6 None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/376fd55b948f49fda100a7da983f285c/roles/d1f99f1b165f4a919c871bd6722ea445` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.674211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ce41c-cc24-4ce6-b85b-4204b50aa928 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:10.675061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24448306-326e-4889-89e8-3bf06f4b65c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=376fd55b948f49fda100a7da983f285c, role_id=d1f99f1b165f4a919c871bd6722ea445)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:10.677019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24448306-326e-4889-89e8-3bf06f4b65c6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:10.694692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 483/950] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:08:10 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/376fd55b948f49fda100a7da983f285c/roles/d1f99f1b165f4a919c871bd6722ea445 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:08:10.715175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baae38-0deb-4868-8531-d4d0c5d89f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:10.715727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baae38-0deb-4868-8531-d4d0c5d89f5c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.716039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baae38-0deb-4868-8531-d4d0c5d89f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.716564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baae38-0deb-4868-8531-d4d0c5d89f5c None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.717222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baae38-0deb-4868-8531-d4d0c5d89f5c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:10.717524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baae38-0deb-4868-8531-d4d0c5d89f5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:10.719172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baae38-0deb-4868-8531-d4d0c5d89f5c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:10.725049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 468/951] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:08:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:10.792841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448306-326e-4889-89e8-3bf06f4b65c6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:10.830757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:10.902915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945c72b-dd68-48c8-b98d-c20712dedd5e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.903963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945c72b-dd68-48c8-b98d-c20712dedd5e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.904452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945c72b-dd68-48c8-b98d-c20712dedd5e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.906430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 469/952] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:10.913854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8a0e9-d491-441b-b0a5-1c501a639b56 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.914142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8a0e9-d491-441b-b0a5-1c501a639b56 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.914971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8a0e9-d491-441b-b0a5-1c501a639b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.928096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24448306-326e-4889-89e8-3bf06f4b65c6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:10.938756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-24a8a0e9-d491-441b-b0a5-1c501a639b56 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:10.940016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-24a8a0e9-d491-441b-b0a5-1c501a639b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:10.970453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:10.970993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:10.971269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:10.971539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:10.972218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:10.973718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:10.976454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WARNING keystone.server.flask.common [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 21 12:08:10.996881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 21 12:08:10.996881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 21 12:08:10.996881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [parameters: {'id': 'e1f18aed940f4e5b8f57100a8e00171e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 21 12:08:10.996881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 21 12:08:10.998581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e1f18aed940f4e5b8f57100a8e00171e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e1f18aed940f4e5b8f57100a8e00171e. Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 12:08:11.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 21 12:08:11.014562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Sep 21 12:08:11.016207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 21 12:08:11.017090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 21 12:08:11.017977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 12:08:11.018762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'id': 'e1f18aed940f4e5b8f57100a8e00171e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:11.020662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:11.021694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e1f18aed940f4e5b8f57100a8e00171e. Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.022829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 484/953] 10.222.0.71 () {60 vars in 1140 bytes} [Wed Sep 21 12:08:10 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:11.025283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:11.037362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 470/954] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:08:11.122523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:11.140909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c84f5-81bf-4a7c-804f-fb9a4d2aedbc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:11.178354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec4d3e-6f19-49a5-a502-fa627aa622f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:11.178997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec4d3e-6f19-49a5-a502-fa627aa622f5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:11.179276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec4d3e-6f19-49a5-a502-fa627aa622f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:11.179551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec4d3e-6f19-49a5-a502-fa627aa622f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:11.187748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec4d3e-6f19-49a5-a502-fa627aa622f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:11.189432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdec4d3e-6f19-49a5-a502-fa627aa622f5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bdec4d3e-6f19-49a5-a502-fa627aa622f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:08:11.201691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:11.202752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:11.204110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:08:11.204110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:08:11.204110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:08:11.204110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:08:11.204110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:08:11.204110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:08:11.204110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:11.204110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.204110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 485/955] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:11.208584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8a0e9-d491-441b-b0a5-1c501a639b56 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:11.312770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8a0e9-d491-441b-b0a5-1c501a639b56 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:11.348173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5d0a4-a2d8-4fea-b63d-a3cf1151510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:11.348838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5d0a4-a2d8-4fea-b63d-a3cf1151510f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:11.349119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5d0a4-a2d8-4fea-b63d-a3cf1151510f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:11.349390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5d0a4-a2d8-4fea-b63d-a3cf1151510f None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:11.350065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5d0a4-a2d8-4fea-b63d-a3cf1151510f None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:11.350366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5d0a4-a2d8-4fea-b63d-a3cf1151510f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:11.351863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5d0a4-a2d8-4fea-b63d-a3cf1151510f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:11.353931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec4d3e-6f19-49a5-a502-fa627aa622f5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:11.377309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 471/956] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:08:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:11.475850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdec4d3e-6f19-49a5-a502-fa627aa622f5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:11.519237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e1419-eb04-41ab-aac5-6382a20b1abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:11.519921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e1419-eb04-41ab-aac5-6382a20b1abf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:11.520203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e1419-eb04-41ab-aac5-6382a20b1abf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:11.520474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e1419-eb04-41ab-aac5-6382a20b1abf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:11.521075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e1419-eb04-41ab-aac5-6382a20b1abf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:11.522475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e1419-eb04-41ab-aac5-6382a20b1abf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c55e1419-eb04-41ab-aac5-6382a20b1abf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:11.532977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:11.534114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:11.534977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 486/957] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:08:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:11.541155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5d0a4-a2d8-4fea-b63d-a3cf1151510f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:11.595827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:11.656454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5d0a4-a2d8-4fea-b63d-a3cf1151510f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:11.698277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e041f08-5b3d-42e3-bc78-6133c22706c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:11.698888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041f08-5b3d-42e3-bc78-6133c22706c7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:11.699169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041f08-5b3d-42e3-bc78-6133c22706c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:11.699443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041f08-5b3d-42e3-bc78-6133c22706c7 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:11.700101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e041f08-5b3d-42e3-bc78-6133c22706c7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:11.700401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e041f08-5b3d-42e3-bc78-6133c22706c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:11.701772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e041f08-5b3d-42e3-bc78-6133c22706c7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:11.711735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 472/958] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:11.713110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e1419-eb04-41ab-aac5-6382a20b1abf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:11.727667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:11.829146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e1419-eb04-41ab-aac5-6382a20b1abf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:11.866165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2da07-c207-41b1-8c61-fd825795cd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:11.866692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2da07-c207-41b1-8c61-fd825795cd66 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:11.866968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2da07-c207-41b1-8c61-fd825795cd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:11.867238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2da07-c207-41b1-8c61-fd825795cd66 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:11.867944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2da07-c207-41b1-8c61-fd825795cd66 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:11.868249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2da07-c207-41b1-8c61-fd825795cd66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:11.869777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2da07-c207-41b1-8c61-fd825795cd66 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:11.878655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 487/959] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:11.885046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e041f08-5b3d-42e3-bc78-6133c22706c7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:11.987945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e041f08-5b3d-42e3-bc78-6133c22706c7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:12.000381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:12.023243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911a9b1-3904-4210-9b3f-71e6d9ff2609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:12.024399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911a9b1-3904-4210-9b3f-71e6d9ff2609 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:12.024672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911a9b1-3904-4210-9b3f-71e6d9ff2609 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:12.024936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911a9b1-3904-4210-9b3f-71e6d9ff2609 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:12.025607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911a9b1-3904-4210-9b3f-71e6d9ff2609 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:12.026969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911a9b1-3904-4210-9b3f-71e6d9ff2609 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:12.050014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 473/960] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:08:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:13.016517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cc586-0347-434e-99fb-76b9a20e5cb0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.017355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cc586-0347-434e-99fb-76b9a20e5cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.017775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cc586-0347-434e-99fb-76b9a20e5cb0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.019484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 488/961] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:13.025138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabce3e-e23d-4a70-a4f6-cccfb5ffaa72 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.025878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabce3e-e23d-4a70-a4f6-cccfb5ffaa72 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.026328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabce3e-e23d-4a70-a4f6-cccfb5ffaa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.044807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fcabce3e-e23d-4a70-a4f6-cccfb5ffaa72 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:13.046214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fcabce3e-e23d-4a70-a4f6-cccfb5ffaa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:13.104330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802befaa-0c5d-44da-8048-d57c73c95724 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.104824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802befaa-0c5d-44da-8048-d57c73c95724 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.105309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802befaa-0c5d-44da-8048-d57c73c95724 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.114914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 489/962] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:13 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:13.117116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d96ca-1c73-4420-b64a-0c8a57369e00 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.117436 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d96ca-1c73-4420-b64a-0c8a57369e00 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.117706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d96ca-1c73-4420-b64a-0c8a57369e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.127517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 474/963] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:08:13.138305 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1b4d96ca-1c73-4420-b64a-0c8a57369e00 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:13.139847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1b4d96ca-1c73-4420-b64a-0c8a57369e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:13.211165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 490/964] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:13.399886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabce3e-e23d-4a70-a4f6-cccfb5ffaa72 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:13.491969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d96ca-1c73-4420-b64a-0c8a57369e00 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:13.494244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:13.500854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:13.510467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabce3e-e23d-4a70-a4f6-cccfb5ffaa72 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:13.546055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee311e-fe05-4516-a5b8-7074159a2c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:13.546821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee311e-fe05-4516-a5b8-7074159a2c4e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.547029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee311e-fe05-4516-a5b8-7074159a2c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.547277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee311e-fe05-4516-a5b8-7074159a2c4e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.558519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee311e-fe05-4516-a5b8-7074159a2c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b9ee311e-fe05-4516-a5b8-7074159a2c4e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:13.563049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:13.564014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:13.564870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 475/965] 10.222.0.71 () {56 vars in 1120 bytes} [Wed Sep 21 12:08:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:13.570046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee311e-fe05-4516-a5b8-7074159a2c4e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:13.614241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d96ca-1c73-4420-b64a-0c8a57369e00 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:13.656157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc314ac9-9f23-4eb9-9c88-73730ad3ff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:13.657114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc314ac9-9f23-4eb9-9c88-73730ad3ff36 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.657602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc314ac9-9f23-4eb9-9c88-73730ad3ff36 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.658053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc314ac9-9f23-4eb9-9c88-73730ad3ff36 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.658885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc314ac9-9f23-4eb9-9c88-73730ad3ff36 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:13.660469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc314ac9-9f23-4eb9-9c88-73730ad3ff36 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:13.696371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee311e-fe05-4516-a5b8-7074159a2c4e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:13.696748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 491/966] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:08:13 2022] POST /identity/v3/services => generated 242 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:08:13.730615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cd0d5-40c7-42f6-98ae-bec097872f20 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.731127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cd0d5-40c7-42f6-98ae-bec097872f20 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.731574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cd0d5-40c7-42f6-98ae-bec097872f20 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.733516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 492/967] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:13.739965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a10f-8616-44c7-968c-eaddef2402d5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.740540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a10f-8616-44c7-968c-eaddef2402d5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.740789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce951cc-74a2-4c8b-8577-1621e7e213d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:13.741414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce951cc-74a2-4c8b-8577-1621e7e213d4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.745118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce951cc-74a2-4c8b-8577-1621e7e213d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.745118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce951cc-74a2-4c8b-8577-1621e7e213d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.745118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce951cc-74a2-4c8b-8577-1621e7e213d4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:13.745118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce951cc-74a2-4c8b-8577-1621e7e213d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:13.745118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce951cc-74a2-4c8b-8577-1621e7e213d4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:13.748324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a73a10f-8616-44c7-968c-eaddef2402d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.754453 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 476/968] 10.222.0.71 () {56 vars in 1115 bytes} [Wed Sep 21 12:08:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:13.762689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce951cc-74a2-4c8b-8577-1621e7e213d4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:13.777324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6a73a10f-8616-44c7-968c-eaddef2402d5 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:13.778144 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:13.780411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6a73a10f-8616-44c7-968c-eaddef2402d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:13.851089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:13.867997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 493/969] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:08:13.904868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce951cc-74a2-4c8b-8577-1621e7e213d4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:13.944091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85257dbe-5242-4b27-9a43-7e24b4562659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:13.944708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85257dbe-5242-4b27-9a43-7e24b4562659 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:13.944996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85257dbe-5242-4b27-9a43-7e24b4562659 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:13.945265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85257dbe-5242-4b27-9a43-7e24b4562659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:13.958334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85257dbe-5242-4b27-9a43-7e24b4562659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:13.959789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85257dbe-5242-4b27-9a43-7e24b4562659 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-85257dbe-5242-4b27-9a43-7e24b4562659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:13.975744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:13.977318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:13.978223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:13.978223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:13.978223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:13.978223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 477/970] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:13.984959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a10f-8616-44c7-968c-eaddef2402d5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:14.089587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a73a10f-8616-44c7-968c-eaddef2402d5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:14.126007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d809d-33fb-4104-be0b-4695973f7a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:14.126621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d809d-33fb-4104-be0b-4695973f7a00 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:14.126903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d809d-33fb-4104-be0b-4695973f7a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:14.127181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d809d-33fb-4104-be0b-4695973f7a00 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:14.127869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d809d-33fb-4104-be0b-4695973f7a00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:14.128180 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d809d-33fb-4104-be0b-4695973f7a00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:14.129563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d809d-33fb-4104-be0b-4695973f7a00 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:14.142294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 494/971] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:14.147867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85257dbe-5242-4b27-9a43-7e24b4562659 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:14.156635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d809d-33fb-4104-be0b-4695973f7a00 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:14.224404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:14.226184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:14.281085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85257dbe-5242-4b27-9a43-7e24b4562659 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:14.288464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d809d-33fb-4104-be0b-4695973f7a00 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:14.322785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff2cb0-194a-4269-8f11-bde5698d92b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:14.323440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff2cb0-194a-4269-8f11-bde5698d92b0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:14.323859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff2cb0-194a-4269-8f11-bde5698d92b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:14.324188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff2cb0-194a-4269-8f11-bde5698d92b0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:14.335041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff2cb0-194a-4269-8f11-bde5698d92b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:14.337126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff2cb0-194a-4269-8f11-bde5698d92b0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:14.346246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5946d9-1491-4364-812f-5f8f73dcb6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:14.347288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5946d9-1491-4364-812f-5f8f73dcb6e6 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:14.347777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5946d9-1491-4364-812f-5f8f73dcb6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:14.348119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5946d9-1491-4364-812f-5f8f73dcb6e6 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:14.348775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5946d9-1491-4364-812f-5f8f73dcb6e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:14.350223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5946d9-1491-4364-812f-5f8f73dcb6e6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-6fff2cb0-194a-4269-8f11-bde5698d92b0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:08:14.350343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:14.351208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 12:08:14.352905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:14.356402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 478/972] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:08:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:14.357884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97d37f-59ec-4b33-b379-b9de3446be63 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:14.358173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97d37f-59ec-4b33-b379-b9de3446be63 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:14.358447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97d37f-59ec-4b33-b379-b9de3446be63 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:14.359802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 479/973] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:14.360429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97d37f-59ec-4b33-b379-b9de3446be63 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:14.367200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.sql.core [None req-bb5946d9-1491-4364-812f-5f8f73dcb6e6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 21 12:08:14.367200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 21 12:08:14.367200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [parameters: {'user_id': '10caa1957c114025973f303ce9104b91', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 21 12:08:14.367200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-bb5946d9-1491-4364-812f-5f8f73dcb6e6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 21 12:08:14.376461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 12:08:14.377370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application yield resource Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application next(self.gen) Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.rollback() Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.force_reraise() Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise self.value Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application session.commit() Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 21 12:08:14.378354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._prepare_impl() Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.session.flush() Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._flush(objects) Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application compat.raise_( Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application flush_context.execute() Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rec.execute(self) Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 21 12:08:14.379357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 12:08:14.380312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application conn.query(q) Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application result.read() Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 12:08:14.381167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [parameters: {'user_id': '10caa1957c114025973f303ce9104b91', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:14.383404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 21 12:08:14.384603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 495/974] 10.222.0.71 () {60 vars in 1131 bytes} [Wed Sep 21 12:08:14 2022] POST /identity/v3/users => generated 160 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193db75-ace9-4258-b0f2-f017b8f0058d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193db75-ace9-4258-b0f2-f017b8f0058d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:14.385739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193db75-ace9-4258-b0f2-f017b8f0058d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:14.406066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a193db75-ace9-4258-b0f2-f017b8f0058d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:14.407691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a193db75-ace9-4258-b0f2-f017b8f0058d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:14.479462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97d37f-59ec-4b33-b379-b9de3446be63 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:14.481432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 496/975] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:14.484714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193db75-ace9-4258-b0f2-f017b8f0058d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:14.510672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:14.524726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c202b6-5962-43c7-9cf8-41498565478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:14.525356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c202b6-5962-43c7-9cf8-41498565478c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:14.525736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c202b6-5962-43c7-9cf8-41498565478c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:14.526180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c202b6-5962-43c7-9cf8-41498565478c None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:14.527019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c202b6-5962-43c7-9cf8-41498565478c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:14.527429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c202b6-5962-43c7-9cf8-41498565478c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:14.529069 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c202b6-5962-43c7-9cf8-41498565478c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:14.537294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:14.558215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 480/976] 10.222.0.71 () {56 vars in 1149 bytes} [Wed Sep 21 12:08:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:14.559277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c202b6-5962-43c7-9cf8-41498565478c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:14.598943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193db75-ace9-4258-b0f2-f017b8f0058d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:14.636015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41036e3-87b4-41c0-b622-ebdf696abe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:14.636985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41036e3-87b4-41c0-b622-ebdf696abe47 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:14.637396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41036e3-87b4-41c0-b622-ebdf696abe47 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:14.637863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41036e3-87b4-41c0-b622-ebdf696abe47 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:14.645787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41036e3-87b4-41c0-b622-ebdf696abe47 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:14.647670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41036e3-87b4-41c0-b622-ebdf696abe47 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a41036e3-87b4-41c0-b622-ebdf696abe47 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:08:14.661037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:14.662132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 12:08:14.663262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:14.670840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 497/977] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:08:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:14.671694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c202b6-5962-43c7-9cf8-41498565478c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:14.672142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41036e3-87b4-41c0-b622-ebdf696abe47 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:14.709769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f651d-3762-4958-a6e5-64876ecd35f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:14.711706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f651d-3762-4958-a6e5-64876ecd35f7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:14.711706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f651d-3762-4958-a6e5-64876ecd35f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:14.711936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f651d-3762-4958-a6e5-64876ecd35f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:14.774730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:14.790167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41036e3-87b4-41c0-b622-ebdf696abe47 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:14.793348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:14.800329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f651d-3762-4958-a6e5-64876ecd35f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:14.801779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f651d-3762-4958-a6e5-64876ecd35f7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:14.828027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32ab76-4bb7-4709-a680-a15a7cc79d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:14.828629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32ab76-4bb7-4709-a680-a15a7cc79d8b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:14.828914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32ab76-4bb7-4709-a680-a15a7cc79d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:14.829187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b32ab76-4bb7-4709-a680-a15a7cc79d8b None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:14.829887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32ab76-4bb7-4709-a680-a15a7cc79d8b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:14.830193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32ab76-4bb7-4709-a680-a15a7cc79d8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:14.831611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b32ab76-4bb7-4709-a680-a15a7cc79d8b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:14.870638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 498/978] 10.222.0.71 () {56 vars in 1101 bytes} [Wed Sep 21 12:08:14 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:14.871494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32ab76-4bb7-4709-a680-a15a7cc79d8b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:14.930687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 481/979] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:14 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:08:14.936409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f651d-3762-4958-a6e5-64876ecd35f7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:15.005719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b32ab76-4bb7-4709-a680-a15a7cc79d8b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.048544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:15.051539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e65bf-30df-45c2-af8c-d20bd539c7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.052250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e65bf-30df-45c2-af8c-d20bd539c7da None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.052575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e65bf-30df-45c2-af8c-d20bd539c7da None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.053802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e65bf-30df-45c2-af8c-d20bd539c7da None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.054283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e65bf-30df-45c2-af8c-d20bd539c7da None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.057443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e65bf-30df-45c2-af8c-d20bd539c7da None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-3e2e65bf-30df-45c2-af8c-d20bd539c7da None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:15.067332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:15.068515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:15.069356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 499/980] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:08:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:15.075707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e65bf-30df-45c2-af8c-d20bd539c7da None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:15.086017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f651d-3762-4958-a6e5-64876ecd35f7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.106385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:15.134009 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e1e05-36c0-45c2-b4cf-d464d7c82a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.135222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e1e05-36c0-45c2-b4cf-d464d7c82a6e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.135504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e1e05-36c0-45c2-b4cf-d464d7c82a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.135795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e1e05-36c0-45c2-b4cf-d464d7c82a6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.147121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e1e05-36c0-45c2-b4cf-d464d7c82a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.150132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e1e05-36c0-45c2-b4cf-d464d7c82a6e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-b17e1e05-36c0-45c2-b4cf-d464d7c82a6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:15.168951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:15.169938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:15.170810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:15.170810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:15.170810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:15.170810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 482/981] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:15.175992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e1e05-36c0-45c2-b4cf-d464d7c82a6e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:15.205565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e65bf-30df-45c2-af8c-d20bd539c7da None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.242893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb5d3e-5f54-4419-ae63-51cf796ede39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.243498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5d3e-5f54-4419-ae63-51cf796ede39 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.243800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5d3e-5f54-4419-ae63-51cf796ede39 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.244081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb5d3e-5f54-4419-ae63-51cf796ede39 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.244710 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb5d3e-5f54-4419-ae63-51cf796ede39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:15.245017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb5d3e-5f54-4419-ae63-51cf796ede39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.246395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bb5d3e-5f54-4419-ae63-51cf796ede39 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.251774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 500/982] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:15.257172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb5d3e-5f54-4419-ae63-51cf796ede39 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:15.292962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e1e05-36c0-45c2-b4cf-d464d7c82a6e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.326406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:15.332760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09c8ea-ad11-400b-b9b5-6cf77e4ced65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.334172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09c8ea-ad11-400b-b9b5-6cf77e4ced65 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.334462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09c8ea-ad11-400b-b9b5-6cf77e4ced65 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.334748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac09c8ea-ad11-400b-b9b5-6cf77e4ced65 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.335398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09c8ea-ad11-400b-b9b5-6cf77e4ced65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:15.335706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09c8ea-ad11-400b-b9b5-6cf77e4ced65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.337661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac09c8ea-ad11-400b-b9b5-6cf77e4ced65 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.351513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 483/983] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:15.357331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09c8ea-ad11-400b-b9b5-6cf77e4ced65 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:15.359468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:15.371750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb5d3e-5f54-4419-ae63-51cf796ede39 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.408250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227b71e-b8cf-4835-8ecc-3e030fd202c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.408860 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227b71e-b8cf-4835-8ecc-3e030fd202c9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.409136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227b71e-b8cf-4835-8ecc-3e030fd202c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.409409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227b71e-b8cf-4835-8ecc-3e030fd202c9 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.410041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227b71e-b8cf-4835-8ecc-3e030fd202c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:15.410344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227b71e-b8cf-4835-8ecc-3e030fd202c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.411706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227b71e-b8cf-4835-8ecc-3e030fd202c9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.421930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 501/984] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:15.423588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227b71e-b8cf-4835-8ecc-3e030fd202c9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:15.469444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac09c8ea-ad11-400b-b9b5-6cf77e4ced65 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.507416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676e518-28fa-483a-93ca-baa044d4e48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.508604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676e518-28fa-483a-93ca-baa044d4e48a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.508881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676e518-28fa-483a-93ca-baa044d4e48a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.509152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676e518-28fa-483a-93ca-baa044d4e48a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.517187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676e518-28fa-483a-93ca-baa044d4e48a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.518634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676e518-28fa-483a-93ca-baa044d4e48a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.528332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227b71e-b8cf-4835-8ecc-3e030fd202c9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9676e518-28fa-483a-93ca-baa044d4e48a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:08:15.530746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:15.531967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:15.533084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:08:15.533084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:08:15.533084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:08:15.533084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:08:15.533084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:08:15.533084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:08:15.533084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:15.533084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:15.533084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 484/985] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:15.539430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676e518-28fa-483a-93ca-baa044d4e48a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:15.568716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0d515-770e-43cc-8aa7-16aa2ff2fdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.569358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0d515-770e-43cc-8aa7-16aa2ff2fdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.569592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0d515-770e-43cc-8aa7-16aa2ff2fdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.569855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0d515-770e-43cc-8aa7-16aa2ff2fdf8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.571083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0d515-770e-43cc-8aa7-16aa2ff2fdf8 None admin] RBAC: Adding query filter params (interface=admin, service_id=b4f3e8c93dae4709bcd945e068f1f207, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:15.571392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0d515-770e-43cc-8aa7-16aa2ff2fdf8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.572994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0d515-770e-43cc-8aa7-16aa2ff2fdf8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.582566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 502/986] 10.222.0.71 () {56 vars in 1249 bytes} [Wed Sep 21 12:08:15 2022] GET /identity/v3/endpoints?service_id=b4f3e8c93dae4709bcd945e068f1f207&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:15.583500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0d515-770e-43cc-8aa7-16aa2ff2fdf8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:15.597377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:15.606118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:15.651101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676e518-28fa-483a-93ca-baa044d4e48a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.687994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fbe2-a2b5-431f-a39c-f0a34d15e502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.688649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fbe2-a2b5-431f-a39c-f0a34d15e502 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.688937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fbe2-a2b5-431f-a39c-f0a34d15e502 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.689213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fbe2-a2b5-431f-a39c-f0a34d15e502 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.689899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970fbe2-a2b5-431f-a39c-f0a34d15e502 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:15.690224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970fbe2-a2b5-431f-a39c-f0a34d15e502 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.691633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970fbe2-a2b5-431f-a39c-f0a34d15e502 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.693372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0d515-770e-43cc-8aa7-16aa2ff2fdf8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.706912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 485/987] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:15.713830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fbe2-a2b5-431f-a39c-f0a34d15e502 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:15.730848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954f87c-57f7-4646-bdac-824af0a2f901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.731454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954f87c-57f7-4646-bdac-824af0a2f901 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.731734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954f87c-57f7-4646-bdac-824af0a2f901 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.732048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954f87c-57f7-4646-bdac-824af0a2f901 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.732646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954f87c-57f7-4646-bdac-824af0a2f901 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.734006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954f87c-57f7-4646-bdac-824af0a2f901 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.747799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 503/988] 10.222.0.71 () {56 vars in 1087 bytes} [Wed Sep 21 12:08:15 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:08:15.838176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fbe2-a2b5-431f-a39c-f0a34d15e502 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:15.853333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:15.876594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc124d-c322-4dfb-aaa8-646de10bade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:15.877212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc124d-c322-4dfb-aaa8-646de10bade6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:15.877490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc124d-c322-4dfb-aaa8-646de10bade6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:15.877759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc124d-c322-4dfb-aaa8-646de10bade6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:15.878496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc124d-c322-4dfb-aaa8-646de10bade6 None admin] RBAC: Adding query filter params (role.id=d1f99f1b165f4a919c871bd6722ea445, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=376fd55b948f49fda100a7da983f285c) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:15.878804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc124d-c322-4dfb-aaa8-646de10bade6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:15.880332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc124d-c322-4dfb-aaa8-646de10bade6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:15.887354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 486/989] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:15 2022] GET /identity/v3/role_assignments?user.id=376fd55b948f49fda100a7da983f285c&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=d1f99f1b165f4a919c871bd6722ea445 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:16.050695 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f146b1-de4b-4067-8990-0ad124e53d5b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:16.051033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f146b1-de4b-4067-8990-0ad124e53d5b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:16.051257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f146b1-de4b-4067-8990-0ad124e53d5b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:16.052839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 504/990] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:16.058952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543673f-db42-40d5-bd28-b145dc3a3b27 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:16.059241 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543673f-db42-40d5-bd28-b145dc3a3b27 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:16.059512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543673f-db42-40d5-bd28-b145dc3a3b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:16.079549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7543673f-db42-40d5-bd28-b145dc3a3b27 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:16.080799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7543673f-db42-40d5-bd28-b145dc3a3b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:16.163157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 487/991] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:16.433261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f146b1-de4b-4067-8990-0ad124e53d5b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:16.520247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:16.541853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f146b1-de4b-4067-8990-0ad124e53d5b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:16.575397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e59e1-9fb8-4567-9318-3fe23727efe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:16.576056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e59e1-9fb8-4567-9318-3fe23727efe5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:16.576340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e59e1-9fb8-4567-9318-3fe23727efe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:16.576611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e59e1-9fb8-4567-9318-3fe23727efe5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:16.577193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e59e1-9fb8-4567-9318-3fe23727efe5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:16.578586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e59e1-9fb8-4567-9318-3fe23727efe5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4d6e59e1-9fb8-4567-9318-3fe23727efe5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:16.588138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:16.589206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 12:08:16.590578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:16.596459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 505/992] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:16.596615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543673f-db42-40d5-bd28-b145dc3a3b27 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:16.702008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543673f-db42-40d5-bd28-b145dc3a3b27 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:16.711473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:16.739843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53114c-9f27-43ec-98dd-ad037e6dd97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:16.740457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53114c-9f27-43ec-98dd-ad037e6dd97c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:16.740734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53114c-9f27-43ec-98dd-ad037e6dd97c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:16.741003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53114c-9f27-43ec-98dd-ad037e6dd97c None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:16.744465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53114c-9f27-43ec-98dd-ad037e6dd97c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:16.744465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53114c-9f27-43ec-98dd-ad037e6dd97c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:16.744465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53114c-9f27-43ec-98dd-ad037e6dd97c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:16.755478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 488/993] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:08:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:16.756743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e59e1-9fb8-4567-9318-3fe23727efe5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:16.866459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e59e1-9fb8-4567-9318-3fe23727efe5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:16.902618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29d0c6-b8de-40d7-944a-adaa63a76db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:16.903226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29d0c6-b8de-40d7-944a-adaa63a76db0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:16.903506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29d0c6-b8de-40d7-944a-adaa63a76db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:16.903920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29d0c6-b8de-40d7-944a-adaa63a76db0 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:16.904556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29d0c6-b8de-40d7-944a-adaa63a76db0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:16.904857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29d0c6-b8de-40d7-944a-adaa63a76db0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:16.906221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29d0c6-b8de-40d7-944a-adaa63a76db0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:16.916363 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 506/994] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:08:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:16.917490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53114c-9f27-43ec-98dd-ad037e6dd97c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:17.017711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53114c-9f27-43ec-98dd-ad037e6dd97c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:17.052256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e56f64-995c-4dac-9dbf-ff4b5043b373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:17.052859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e56f64-995c-4dac-9dbf-ff4b5043b373 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:17.053133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e56f64-995c-4dac-9dbf-ff4b5043b373 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:17.053398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e56f64-995c-4dac-9dbf-ff4b5043b373 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:17.054085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e56f64-995c-4dac-9dbf-ff4b5043b373 None admin] RBAC: Adding query filter params (interface=public, service_id=28284e6c88444f5993f24dd6bbd2ef4b, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:17.054383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e56f64-995c-4dac-9dbf-ff4b5043b373 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:17.056307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e56f64-995c-4dac-9dbf-ff4b5043b373 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:17.066173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 489/995] 10.222.0.71 () {56 vars in 1251 bytes} [Wed Sep 21 12:08:16 2022] GET /identity/v3/endpoints?service_id=28284e6c88444f5993f24dd6bbd2ef4b&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:17.067738 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29d0c6-b8de-40d7-944a-adaa63a76db0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:17.080560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:17.173244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29d0c6-b8de-40d7-944a-adaa63a76db0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:17.208702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c862fbb0-9898-4f1f-b413-dd484fcff8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:17.209311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862fbb0-9898-4f1f-b413-dd484fcff8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:17.209594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862fbb0-9898-4f1f-b413-dd484fcff8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:17.209867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862fbb0-9898-4f1f-b413-dd484fcff8e0 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:17.210472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862fbb0-9898-4f1f-b413-dd484fcff8e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:17.211897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862fbb0-9898-4f1f-b413-dd484fcff8e0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:17.218414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 507/996] 10.222.0.71 () {56 vars in 1087 bytes} [Wed Sep 21 12:08:17 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:08:17.987980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc89cc-88bc-467d-9c9e-7156747aefb5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:17.988519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc89cc-88bc-467d-9c9e-7156747aefb5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:17.988519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc89cc-88bc-467d-9c9e-7156747aefb5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:17.990087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 490/997] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:17.997843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bba96-f88f-42aa-88d9-70ce3e1bdef3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:17.998347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bba96-f88f-42aa-88d9-70ce3e1bdef3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:17.998424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bba96-f88f-42aa-88d9-70ce3e1bdef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:18.017778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-139bba96-f88f-42aa-88d9-70ce3e1bdef3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:18.018892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-139bba96-f88f-42aa-88d9-70ce3e1bdef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:18.096541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 508/998] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:18.102076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68180ae-c4b5-4260-9881-e950699c4c58 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:18.102355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68180ae-c4b5-4260-9881-e950699c4c58 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:18.102622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68180ae-c4b5-4260-9881-e950699c4c58 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:18.104094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 491/999] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:18.109406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0c3ee-bd11-403f-9c00-81ac6222275c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:18.109687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0c3ee-bd11-403f-9c00-81ac6222275c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:18.109950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0c3ee-bd11-403f-9c00-81ac6222275c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:18.117992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:18.133634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c7f0c3ee-bd11-403f-9c00-81ac6222275c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:18.134840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c7f0c3ee-bd11-403f-9c00-81ac6222275c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:18.202450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 509/1000] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:18.361160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68180ae-c4b5-4260-9881-e950699c4c58 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:18.402006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:18.469055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0c3ee-bd11-403f-9c00-81ac6222275c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:18.474575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68180ae-c4b5-4260-9881-e950699c4c58 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:18.509909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df0f44-b208-4702-82be-1f6357efb37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:18.510540 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df0f44-b208-4702-82be-1f6357efb37d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:18.510812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df0f44-b208-4702-82be-1f6357efb37d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:18.511367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df0f44-b208-4702-82be-1f6357efb37d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:18.511643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df0f44-b208-4702-82be-1f6357efb37d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:18.513024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df0f44-b208-4702-82be-1f6357efb37d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-a6df0f44-b208-4702-82be-1f6357efb37d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:18.522363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:18.524836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:18.525996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 492/1001] 10.222.0.71 () {56 vars in 1126 bytes} [Wed Sep 21 12:08:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:18.530230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df0f44-b208-4702-82be-1f6357efb37d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:18.582051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0c3ee-bd11-403f-9c00-81ac6222275c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:18.617925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ac8b0-01d4-4dac-82fa-c2d2980c661c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:18.618546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ac8b0-01d4-4dac-82fa-c2d2980c661c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:18.618834 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ac8b0-01d4-4dac-82fa-c2d2980c661c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:18.619109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ac8b0-01d4-4dac-82fa-c2d2980c661c None admin] PATH_INFO: `/v3/services/image` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:18.619698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ac8b0-01d4-4dac-82fa-c2d2980c661c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:18.621112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ac8b0-01d4-4dac-82fa-c2d2980c661c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:18.623121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-af9ac8b0-01d4-4dac-82fa-c2d2980c661c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:18.631805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:18.633327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 12:08:18.634202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:18.634972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:18.636647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 510/1002] 10.222.0.71 () {56 vars in 1105 bytes} [Wed Sep 21 12:08:18 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:18.639957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df0f44-b208-4702-82be-1f6357efb37d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:18.642434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ac8b0-01d4-4dac-82fa-c2d2980c661c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:18.675835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d8030-b938-49f5-9776-c4da6ce67856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:18.676441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d8030-b938-49f5-9776-c4da6ce67856 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:18.676726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d8030-b938-49f5-9776-c4da6ce67856 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:18.676998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937d8030-b938-49f5-9776-c4da6ce67856 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:18.677623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d8030-b938-49f5-9776-c4da6ce67856 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:18.677927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d8030-b938-49f5-9776-c4da6ce67856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:18.679283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937d8030-b938-49f5-9776-c4da6ce67856 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:18.684480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 493/1003] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:18.690276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d8030-b938-49f5-9776-c4da6ce67856 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:18.750338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ac8b0-01d4-4dac-82fa-c2d2980c661c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:18.789310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec645f7f-6487-4158-8801-eec11e9bd0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:18.789914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec645f7f-6487-4158-8801-eec11e9bd0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:18.790189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec645f7f-6487-4158-8801-eec11e9bd0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:18.790454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec645f7f-6487-4158-8801-eec11e9bd0e2 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:18.791068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec645f7f-6487-4158-8801-eec11e9bd0e2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:18.791365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec645f7f-6487-4158-8801-eec11e9bd0e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:18.792768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec645f7f-6487-4158-8801-eec11e9bd0e2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:18.797783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 511/1004] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:18.799059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937d8030-b938-49f5-9776-c4da6ce67856 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:18.803186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec645f7f-6487-4158-8801-eec11e9bd0e2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:18.837976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ff484-1f3d-4e07-a04f-86ad8752577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:18.838571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ff484-1f3d-4e07-a04f-86ad8752577c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:18.838855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ff484-1f3d-4e07-a04f-86ad8752577c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:18.839291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ff484-1f3d-4e07-a04f-86ad8752577c None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:18.840767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ff484-1f3d-4e07-a04f-86ad8752577c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:18.841086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ff484-1f3d-4e07-a04f-86ad8752577c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:18.842715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ff484-1f3d-4e07-a04f-86ad8752577c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:18.848848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 494/1005] 10.222.0.71 () {56 vars in 1122 bytes} [Wed Sep 21 12:08:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:18.854440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ff484-1f3d-4e07-a04f-86ad8752577c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:18.873516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:18.884316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:18.920148 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec645f7f-6487-4158-8801-eec11e9bd0e2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:18.957394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a051c67-baa5-407f-ba7c-05381958a646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:18.958000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a051c67-baa5-407f-ba7c-05381958a646 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:18.958279 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a051c67-baa5-407f-ba7c-05381958a646 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:18.958550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a051c67-baa5-407f-ba7c-05381958a646 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:18.959174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a051c67-baa5-407f-ba7c-05381958a646 None admin] RBAC: Adding query filter params (type=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:18.959473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a051c67-baa5-407f-ba7c-05381958a646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:18.960895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a051c67-baa5-407f-ba7c-05381958a646 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:18.963630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ff484-1f3d-4e07-a04f-86ad8752577c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:18.965546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 512/1006] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:19.013665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8590a-d7d5-4516-b5fe-5df39cde22e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:19.014285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8590a-d7d5-4516-b5fe-5df39cde22e1 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:19.014759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8590a-d7d5-4516-b5fe-5df39cde22e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:19.015106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8590a-d7d5-4516-b5fe-5df39cde22e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:19.015893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8590a-d7d5-4516-b5fe-5df39cde22e1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:19.017280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8590a-d7d5-4516-b5fe-5df39cde22e1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:19.035679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 495/1007] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:08:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:19.429408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9991c-848b-4c3b-a2d3-909901014ca5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:19.430020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9991c-848b-4c3b-a2d3-909901014ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:19.430100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9991c-848b-4c3b-a2d3-909901014ca5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:19.431444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 513/1008] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:08:19.436888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d202f39-a967-4df6-9f7e-996ce3f2b00a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:19.437247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d202f39-a967-4df6-9f7e-996ce3f2b00a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:19.437596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d202f39-a967-4df6-9f7e-996ce3f2b00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:19.456630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6d202f39-a967-4df6-9f7e-996ce3f2b00a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:19.457823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6d202f39-a967-4df6-9f7e-996ce3f2b00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:19.536605 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 496/1009] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:19 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:19.812888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9991c-848b-4c3b-a2d3-909901014ca5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:19.923578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9991c-848b-4c3b-a2d3-909901014ca5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:19.952130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:19.962137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84333250-0756-4f0e-ace7-424aae33e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:19.962777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84333250-0756-4f0e-ace7-424aae33e626 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:19.963057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84333250-0756-4f0e-ace7-424aae33e626 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:19.963329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84333250-0756-4f0e-ace7-424aae33e626 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:19.963944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84333250-0756-4f0e-ace7-424aae33e626 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:19.965310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84333250-0756-4f0e-ace7-424aae33e626 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-84333250-0756-4f0e-ace7-424aae33e626 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:19.974449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:19.975551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 12:08:19.977290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:19.980076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 514/1010] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:19.982115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d202f39-a967-4df6-9f7e-996ce3f2b00a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:19.996750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:20.082638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d202f39-a967-4df6-9f7e-996ce3f2b00a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:20.116720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4347-0cd3-4cd7-8f30-dcf49024ac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:20.117935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4347-0cd3-4cd7-8f30-dcf49024ac2b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:20.118210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4347-0cd3-4cd7-8f30-dcf49024ac2b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:20.118516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4347-0cd3-4cd7-8f30-dcf49024ac2b None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:20.119102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4347-0cd3-4cd7-8f30-dcf49024ac2b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:20.119398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4347-0cd3-4cd7-8f30-dcf49024ac2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:20.120861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4347-0cd3-4cd7-8f30-dcf49024ac2b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:20.125817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 497/1011] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:08:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:20.131450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84333250-0756-4f0e-ace7-424aae33e626 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:20.237323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84333250-0756-4f0e-ace7-424aae33e626 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:20.272554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b858907-26a3-4ad2-bc5e-96bf94c20b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:20.273374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b858907-26a3-4ad2-bc5e-96bf94c20b2f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:20.274146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b858907-26a3-4ad2-bc5e-96bf94c20b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:20.274420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b858907-26a3-4ad2-bc5e-96bf94c20b2f None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:20.275051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b858907-26a3-4ad2-bc5e-96bf94c20b2f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:20.275353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b858907-26a3-4ad2-bc5e-96bf94c20b2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:20.276746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b858907-26a3-4ad2-bc5e-96bf94c20b2f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:20.286867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 515/1012] 10.222.0.71 () {56 vars in 1112 bytes} [Wed Sep 21 12:08:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:20.288341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4347-0cd3-4cd7-8f30-dcf49024ac2b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:20.366886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:20.391606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4347-0cd3-4cd7-8f30-dcf49024ac2b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:20.431207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b637d1fd-80b7-4bb8-871c-98f7c3ebb6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:20.432149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637d1fd-80b7-4bb8-871c-98f7c3ebb6d8 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:20.432631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637d1fd-80b7-4bb8-871c-98f7c3ebb6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:20.433116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637d1fd-80b7-4bb8-871c-98f7c3ebb6d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:20.433983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637d1fd-80b7-4bb8-871c-98f7c3ebb6d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:20.435534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b637d1fd-80b7-4bb8-871c-98f7c3ebb6d8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:20.462396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 498/1013] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:08:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:08:21.189893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48471f-d2f0-4fc6-b182-c07e75a92a5e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:21.190444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48471f-d2f0-4fc6-b182-c07e75a92a5e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:21.190444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48471f-d2f0-4fc6-b182-c07e75a92a5e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:21.191970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 516/1014] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:21.197278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f185fd1-f4c3-4564-9a10-b80e67f95523 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:21.197768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f185fd1-f4c3-4564-9a10-b80e67f95523 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:21.197859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f185fd1-f4c3-4564-9a10-b80e67f95523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:21.218094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8f185fd1-f4c3-4564-9a10-b80e67f95523 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:21.219308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8f185fd1-f4c3-4564-9a10-b80e67f95523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:21.233735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5119161-295f-4a2f-8564-eb01167a06a6 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:21.234063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5119161-295f-4a2f-8564-eb01167a06a6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:21.234287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5119161-295f-4a2f-8564-eb01167a06a6 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:21.235894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 517/1015] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:21.240795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9313fb-6cc4-48e6-9097-dfef342f73d5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:21.241086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9313fb-6cc4-48e6-9097-dfef342f73d5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:21.241363 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9313fb-6cc4-48e6-9097-dfef342f73d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:21.260018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9f9313fb-6cc4-48e6-9097-dfef342f73d5 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:21.261223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9f9313fb-6cc4-48e6-9097-dfef342f73d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:21.300425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 499/1016] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:21 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:21.316041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:21.335604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 518/1017] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:21 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:08:21.571046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f185fd1-f4c3-4564-9a10-b80e67f95523 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:21.620431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9313fb-6cc4-48e6-9097-dfef342f73d5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:21.637311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:21.687462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f185fd1-f4c3-4564-9a10-b80e67f95523 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:21.724899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12faaf-82d9-4a5d-b204-e6ebf422661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:21.725546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12faaf-82d9-4a5d-b204-e6ebf422661d None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:21.725782 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12faaf-82d9-4a5d-b204-e6ebf422661d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:21.726119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12faaf-82d9-4a5d-b204-e6ebf422661d None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:21.726767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12faaf-82d9-4a5d-b204-e6ebf422661d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:21.728162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12faaf-82d9-4a5d-b204-e6ebf422661d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:21.728473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9313fb-6cc4-48e6-9097-dfef342f73d5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:21.751213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 500/1018] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:08:21 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:21.792663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714d336-1f9e-41c2-a7cc-637adc0686b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:21.793315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714d336-1f9e-41c2-a7cc-637adc0686b6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:21.793604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714d336-1f9e-41c2-a7cc-637adc0686b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:21.793877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714d336-1f9e-41c2-a7cc-637adc0686b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:21.795069 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714d336-1f9e-41c2-a7cc-637adc0686b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:21.796551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714d336-1f9e-41c2-a7cc-637adc0686b6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:21.801393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 519/1019] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:21.804114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12faaf-82d9-4a5d-b204-e6ebf422661d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:21.921606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12faaf-82d9-4a5d-b204-e6ebf422661d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:21.941424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:21.961313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd475bf-18ec-435e-9303-6d38d510ca08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:21.961947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd475bf-18ec-435e-9303-6d38d510ca08 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:21.962231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd475bf-18ec-435e-9303-6d38d510ca08 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:21.962504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd475bf-18ec-435e-9303-6d38d510ca08 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:21.963158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd475bf-18ec-435e-9303-6d38d510ca08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:21.964899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd475bf-18ec-435e-9303-6d38d510ca08 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:21.993414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 501/1020] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:08:21 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:22.745665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a28b-15d5-4adc-a660-137ebeb42373 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:22.745665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a28b-15d5-4adc-a660-137ebeb42373 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:22.745665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a28b-15d5-4adc-a660-137ebeb42373 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:22.746880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 520/1021] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:22.752114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685299-47d6-49ad-99bf-f545840452dc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:22.752518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685299-47d6-49ad-99bf-f545840452dc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:22.752687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e685299-47d6-49ad-99bf-f545840452dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:22.774142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3e685299-47d6-49ad-99bf-f545840452dc None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:22.775388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3e685299-47d6-49ad-99bf-f545840452dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:22.853824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 502/1022] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:23.121277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a28b-15d5-4adc-a660-137ebeb42373 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:23.175446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:23.230595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a28b-15d5-4adc-a660-137ebeb42373 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:23.265073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be91e0-9db2-4b74-86d6-26784cf9be81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:23.265700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be91e0-9db2-4b74-86d6-26784cf9be81 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:23.265981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be91e0-9db2-4b74-86d6-26784cf9be81 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:23.266256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be91e0-9db2-4b74-86d6-26784cf9be81 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:23.277306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be91e0-9db2-4b74-86d6-26784cf9be81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-85be91e0-9db2-4b74-86d6-26784cf9be81 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:23.282171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:23.283184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:23.284671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:23.288187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 521/1023] 10.222.0.71 () {56 vars in 1120 bytes} [Wed Sep 21 12:08:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:23.289255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e685299-47d6-49ad-99bf-f545840452dc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:23.387090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:23.391122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e685299-47d6-49ad-99bf-f545840452dc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:23.426373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a4d83-37f8-481b-b50b-21e13ad3702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:23.426982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a4d83-37f8-481b-b50b-21e13ad3702d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:23.427264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a4d83-37f8-481b-b50b-21e13ad3702d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:23.427536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a4d83-37f8-481b-b50b-21e13ad3702d None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:23.428228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a4d83-37f8-481b-b50b-21e13ad3702d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:23.428531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a4d83-37f8-481b-b50b-21e13ad3702d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:23.430565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a4d83-37f8-481b-b50b-21e13ad3702d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:23.443305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 503/1024] 10.222.0.71 () {56 vars in 1115 bytes} [Wed Sep 21 12:08:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:23.445116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be91e0-9db2-4b74-86d6-26784cf9be81 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:23.558007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be91e0-9db2-4b74-86d6-26784cf9be81 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:23.568679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:23.600310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc67757-39b5-4f84-a31a-55f2589a65ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:23.601080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc67757-39b5-4f84-a31a-55f2589a65ed None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:23.601972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc67757-39b5-4f84-a31a-55f2589a65ed None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:23.602241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc67757-39b5-4f84-a31a-55f2589a65ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:23.612667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc67757-39b5-4f84-a31a-55f2589a65ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:23.614119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc67757-39b5-4f84-a31a-55f2589a65ed None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-6cc67757-39b5-4f84-a31a-55f2589a65ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:23.629039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:23.630158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:23.631241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:23.631241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:23.631241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:23.634883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 522/1025] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:23.636254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a4d83-37f8-481b-b50b-21e13ad3702d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:23.760003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a4d83-37f8-481b-b50b-21e13ad3702d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:23.801990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5659cf4-1185-4aac-b321-bf727364490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:23.802713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5659cf4-1185-4aac-b321-bf727364490b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:23.802995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5659cf4-1185-4aac-b321-bf727364490b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:23.803267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5659cf4-1185-4aac-b321-bf727364490b None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:23.803955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5659cf4-1185-4aac-b321-bf727364490b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:23.804573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5659cf4-1185-4aac-b321-bf727364490b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:23.806249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5659cf4-1185-4aac-b321-bf727364490b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:23.819793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 504/1026] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:23.826263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc67757-39b5-4f84-a31a-55f2589a65ed None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:23.933077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc67757-39b5-4f84-a31a-55f2589a65ed None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:23.941789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f61ad-b40d-44aa-844e-132b09da5c22 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:23.942074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f61ad-b40d-44aa-844e-132b09da5c22 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:23.942343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f61ad-b40d-44aa-844e-132b09da5c22 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:23.943685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 505/1027] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:08:23.949533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347eb94-314b-4021-9f76-a34b1dd4993c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:23.949818 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347eb94-314b-4021-9f76-a34b1dd4993c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:23.950089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347eb94-314b-4021-9f76-a34b1dd4993c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:23.969074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a347eb94-314b-4021-9f76-a34b1dd4993c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:23.970265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a347eb94-314b-4021-9f76-a34b1dd4993c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:23.976346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa31547-a0d3-4451-a341-3742704285d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:23.977233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa31547-a0d3-4451-a341-3742704285d3 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:23.978206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa31547-a0d3-4451-a341-3742704285d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:23.978681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa31547-a0d3-4451-a341-3742704285d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:23.987724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:23.989631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa31547-a0d3-4451-a341-3742704285d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:23.991091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa31547-a0d3-4451-a341-3742704285d3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5fa31547-a0d3-4451-a341-3742704285d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:08:24.002867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:24.003884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:24.004878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 523/1028] 10.222.0.71 () {56 vars in 1093 bytes} [Wed Sep 21 12:08:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:24.010179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa31547-a0d3-4451-a341-3742704285d3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.037029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:24.045650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 506/1029] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:08:24.124211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa31547-a0d3-4451-a341-3742704285d3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:24.158788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dbb25-148f-4001-bb6d-ba87df2354ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:24.159399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dbb25-148f-4001-bb6d-ba87df2354ff None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.159676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dbb25-148f-4001-bb6d-ba87df2354ff None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.159976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dbb25-148f-4001-bb6d-ba87df2354ff None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.160702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dbb25-148f-4001-bb6d-ba87df2354ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:24.161005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dbb25-148f-4001-bb6d-ba87df2354ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:24.162422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dbb25-148f-4001-bb6d-ba87df2354ff None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:24.177509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79f0a5-9d3e-4ed3-ba94-bb7e92e59353 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.177805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79f0a5-9d3e-4ed3-ba94-bb7e92e59353 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.178054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79f0a5-9d3e-4ed3-ba94-bb7e92e59353 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.179551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 507/1030] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:08:24.186351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 524/1031] 10.222.0.71 () {56 vars in 1133 bytes} [Wed Sep 21 12:08:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:24.186507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046986c-73a5-4879-a9f4-d89bd549ce53 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.186507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046986c-73a5-4879-a9f4-d89bd549ce53 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.186507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046986c-73a5-4879-a9f4-d89bd549ce53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.187308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dbb25-148f-4001-bb6d-ba87df2354ff None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.202866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7046986c-73a5-4879-a9f4-d89bd549ce53 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:24.204084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7046986c-73a5-4879-a9f4-d89bd549ce53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:24.268315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:24.276099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 508/1032] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:24.292224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dbb25-148f-4001-bb6d-ba87df2354ff None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:24.323003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046986c-73a5-4879-a9f4-d89bd549ce53 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.327007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61732ab6-8863-4cac-893a-6fa3894487b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:24.327613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61732ab6-8863-4cac-893a-6fa3894487b0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.327934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61732ab6-8863-4cac-893a-6fa3894487b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.328205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61732ab6-8863-4cac-893a-6fa3894487b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.400358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61732ab6-8863-4cac-893a-6fa3894487b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:24.402575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61732ab6-8863-4cac-893a-6fa3894487b0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:24.412695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:24.433403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046986c-73a5-4879-a9f4-d89bd549ce53 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:24.470334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b86ba0-f51c-45a7-a20c-1174d78b32f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:24.470983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b86ba0-f51c-45a7-a20c-1174d78b32f6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.471273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b86ba0-f51c-45a7-a20c-1174d78b32f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.471522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b86ba0-f51c-45a7-a20c-1174d78b32f6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.472143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b86ba0-f51c-45a7-a20c-1174d78b32f6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:24.473536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b86ba0-f51c-45a7-a20c-1174d78b32f6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-16b86ba0-f51c-45a7-a20c-1174d78b32f6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:24.482812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:24.483906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:24.484803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 509/1033] 10.222.0.71 () {56 vars in 1105 bytes} [Wed Sep 21 12:08:24 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:24.489944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b86ba0-f51c-45a7-a20c-1174d78b32f6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.490600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:24.520913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 525/1034] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:24 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 12:08:24.522130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61732ab6-8863-4cac-893a-6fa3894487b0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.595113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b86ba0-f51c-45a7-a20c-1174d78b32f6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:24.621562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61732ab6-8863-4cac-893a-6fa3894487b0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:24.629124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced49b0-41e8-4061-a124-d62866452ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:24.629724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced49b0-41e8-4061-a124-d62866452ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.630002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced49b0-41e8-4061-a124-d62866452ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.630273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced49b0-41e8-4061-a124-d62866452ee9 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.630901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced49b0-41e8-4061-a124-d62866452ee9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:24.631199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced49b0-41e8-4061-a124-d62866452ee9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:24.632667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced49b0-41e8-4061-a124-d62866452ee9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:24.637409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 510/1035] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:24.638184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced49b0-41e8-4061-a124-d62866452ee9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.650008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:24.660492 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885718fd-bfc3-4208-84f4-c954c5a0cc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:24.661119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885718fd-bfc3-4208-84f4-c954c5a0cc8b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.661396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885718fd-bfc3-4208-84f4-c954c5a0cc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.661696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885718fd-bfc3-4208-84f4-c954c5a0cc8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.672063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885718fd-bfc3-4208-84f4-c954c5a0cc8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:24.673508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885718fd-bfc3-4208-84f4-c954c5a0cc8b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-885718fd-bfc3-4208-84f4-c954c5a0cc8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:24.688996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:24.690126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:24.695868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:24.695868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:24.695868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:24.695868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 526/1036] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:24.695868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885718fd-bfc3-4208-84f4-c954c5a0cc8b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.721139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:24.743525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced49b0-41e8-4061-a124-d62866452ee9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:24.787609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04c65c-fe58-4a84-bac7-a65d49783f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:24.787609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04c65c-fe58-4a84-bac7-a65d49783f7f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.787609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04c65c-fe58-4a84-bac7-a65d49783f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.787609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04c65c-fe58-4a84-bac7-a65d49783f7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.788586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04c65c-fe58-4a84-bac7-a65d49783f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:24.789770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04c65c-fe58-4a84-bac7-a65d49783f7f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:24.791371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 511/1037] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:24.792159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04c65c-fe58-4a84-bac7-a65d49783f7f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.799353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885718fd-bfc3-4208-84f4-c954c5a0cc8b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:24.833086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5407ab-167e-4a4d-800f-509ff2998ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:24.833769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5407ab-167e-4a4d-800f-509ff2998ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.833954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5407ab-167e-4a4d-800f-509ff2998ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.834220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5407ab-167e-4a4d-800f-509ff2998ca6 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.834842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5407ab-167e-4a4d-800f-509ff2998ca6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:24.835142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5407ab-167e-4a4d-800f-509ff2998ca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:24.837058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5407ab-167e-4a4d-800f-509ff2998ca6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:24.842186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 527/1038] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:24.842957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5407ab-167e-4a4d-800f-509ff2998ca6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.875080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:24.897394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04c65c-fe58-4a84-bac7-a65d49783f7f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:24.932343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fde5e6-85b8-4de1-b026-c4e3978a78d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:24.932946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fde5e6-85b8-4de1-b026-c4e3978a78d5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.933225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fde5e6-85b8-4de1-b026-c4e3978a78d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.933494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fde5e6-85b8-4de1-b026-c4e3978a78d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.934151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fde5e6-85b8-4de1-b026-c4e3978a78d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:24.934433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fde5e6-85b8-4de1-b026-c4e3978a78d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:24.935822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fde5e6-85b8-4de1-b026-c4e3978a78d5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:24.935979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:24.943189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5407ab-167e-4a4d-800f-509ff2998ca6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:24.948488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 512/1039] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:24.949286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fde5e6-85b8-4de1-b026-c4e3978a78d5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:24.977892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4526c-ad1f-4138-a27e-005d4497b0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:24.978491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4526c-ad1f-4138-a27e-005d4497b0f3 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:24.978764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4526c-ad1f-4138-a27e-005d4497b0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:24.979032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4526c-ad1f-4138-a27e-005d4497b0f3 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:24.979667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4526c-ad1f-4138-a27e-005d4497b0f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:24.981112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4526c-ad1f-4138-a27e-005d4497b0f3 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:25.002856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 528/1040] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:08:24 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:25.003680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4526c-ad1f-4138-a27e-005d4497b0f3 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:25.063188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fde5e6-85b8-4de1-b026-c4e3978a78d5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:25.104941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634644f-1a0a-4c5a-b4db-c493b836f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:25.105646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634644f-1a0a-4c5a-b4db-c493b836f008 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:25.111924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634644f-1a0a-4c5a-b4db-c493b836f008 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:25.112209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634644f-1a0a-4c5a-b4db-c493b836f008 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:25.112967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634644f-1a0a-4c5a-b4db-c493b836f008 None admin] RBAC: Adding query filter params (interface=public, service_id=15f47fadc9b04a89b7edef2fc5bde7b1, region_id=RegionOne) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:25.113277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634644f-1a0a-4c5a-b4db-c493b836f008 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:25.114704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634644f-1a0a-4c5a-b4db-c493b836f008 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:25.117978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4526c-ad1f-4138-a27e-005d4497b0f3 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:25.130336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 513/1041] 10.222.0.71 () {56 vars in 1251 bytes} [Wed Sep 21 12:08:24 2022] GET /identity/v3/endpoints?service_id=15f47fadc9b04a89b7edef2fc5bde7b1&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:25.137013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634644f-1a0a-4c5a-b4db-c493b836f008 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:25.140597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:25.162471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8723a9-99c1-4e98-822f-b7ef58d5bd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:25.163396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8723a9-99c1-4e98-822f-b7ef58d5bd2a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:25.164402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8723a9-99c1-4e98-822f-b7ef58d5bd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:25.164872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8723a9-99c1-4e98-822f-b7ef58d5bd2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:25.173491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8723a9-99c1-4e98-822f-b7ef58d5bd2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:25.175367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8723a9-99c1-4e98-822f-b7ef58d5bd2a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-af8723a9-99c1-4e98-822f-b7ef58d5bd2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:08:25.188547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:25.189626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:25.190814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:08:25.190814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:08:25.190814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:08:25.190814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:08:25.190814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:08:25.190814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:08:25.190814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:25.190814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:25.192596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 529/1042] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:25.199721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8723a9-99c1-4e98-822f-b7ef58d5bd2a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:25.213765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:25.266207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634644f-1a0a-4c5a-b4db-c493b836f008 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:25.305769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44588b4b-6ccf-42f0-b115-2a7073df3195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:25.306382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44588b4b-6ccf-42f0-b115-2a7073df3195 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:25.306664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44588b4b-6ccf-42f0-b115-2a7073df3195 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:25.306938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44588b4b-6ccf-42f0-b115-2a7073df3195 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:25.307542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44588b4b-6ccf-42f0-b115-2a7073df3195 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:25.308948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44588b4b-6ccf-42f0-b115-2a7073df3195 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:25.319323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8723a9-99c1-4e98-822f-b7ef58d5bd2a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:25.319640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 514/1043] 10.222.0.71 () {56 vars in 1087 bytes} [Wed Sep 21 12:08:25 2022] GET /identity/v3/services => generated 1957 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:08:25.360600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa671d2-0d56-4a93-b398-38279b28a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:25.361225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa671d2-0d56-4a93-b398-38279b28a253 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:25.361510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa671d2-0d56-4a93-b398-38279b28a253 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:25.361843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa671d2-0d56-4a93-b398-38279b28a253 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:25.362645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa671d2-0d56-4a93-b398-38279b28a253 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:25.362991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa671d2-0d56-4a93-b398-38279b28a253 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:25.364991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa671d2-0d56-4a93-b398-38279b28a253 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:25.384553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 530/1044] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:08:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:25.385997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44588b4b-6ccf-42f0-b115-2a7073df3195 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:25.462152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:25.503023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44588b4b-6ccf-42f0-b115-2a7073df3195 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:25.546224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98419cf-e1df-4ffc-a551-5b856b5a5c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:25.546831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98419cf-e1df-4ffc-a551-5b856b5a5c87 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:25.547111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98419cf-e1df-4ffc-a551-5b856b5a5c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:25.547382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98419cf-e1df-4ffc-a551-5b856b5a5c87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:25.548151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98419cf-e1df-4ffc-a551-5b856b5a5c87 None admin] RBAC: Adding query filter params (role.id=d1f99f1b165f4a919c871bd6722ea445, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=463487449c1d4aada1b5f279c959b0e6) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:25.548460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98419cf-e1df-4ffc-a551-5b856b5a5c87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:25.549894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98419cf-e1df-4ffc-a551-5b856b5a5c87 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:25.556472 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 515/1045] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:25 2022] GET /identity/v3/role_assignments?user.id=463487449c1d4aada1b5f279c959b0e6&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=d1f99f1b165f4a919c871bd6722ea445 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:27.252824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0db11-b405-46fa-ba83-147a6a37af6c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:27.253349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0db11-b405-46fa-ba83-147a6a37af6c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:27.253468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0db11-b405-46fa-ba83-147a6a37af6c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:27.254722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 531/1046] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:27.262070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d80f3-c896-4557-bb83-3cc4da296196 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:27.262508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d80f3-c896-4557-bb83-3cc4da296196 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:27.262600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d80f3-c896-4557-bb83-3cc4da296196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:27.282311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a32d80f3-c896-4557-bb83-3cc4da296196 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:27.283518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a32d80f3-c896-4557-bb83-3cc4da296196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:27.354642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 516/1047] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:27.510690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03190106-a17e-424c-89ee-d3c680007e5c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:27.510879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03190106-a17e-424c-89ee-d3c680007e5c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:27.511162 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03190106-a17e-424c-89ee-d3c680007e5c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:27.512666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 532/1048] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:27.517934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20986286-23b5-427d-8f33-4f69489aadc1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:27.518229 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20986286-23b5-427d-8f33-4f69489aadc1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:27.518503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20986286-23b5-427d-8f33-4f69489aadc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:27.537123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-20986286-23b5-427d-8f33-4f69489aadc1 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:27.538307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-20986286-23b5-427d-8f33-4f69489aadc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:27.598026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:27.611439 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 517/1049] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:27.630267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03190106-a17e-424c-89ee-d3c680007e5c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:27.703689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f6766-426f-4a46-8b8e-20da732e4050 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:27.704006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f6766-426f-4a46-8b8e-20da732e4050 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:27.704278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955f6766-426f-4a46-8b8e-20da732e4050 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:27.705622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 518/1050] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:08:27.708952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:27.712864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e79f96-eb0a-4b0e-82ed-c8f8b452dee6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:27.713379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e79f96-eb0a-4b0e-82ed-c8f8b452dee6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:27.713894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e79f96-eb0a-4b0e-82ed-c8f8b452dee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:27.736540 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-06e79f96-eb0a-4b0e-82ed-c8f8b452dee6 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:27.738475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-06e79f96-eb0a-4b0e-82ed-c8f8b452dee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:27.742109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03190106-a17e-424c-89ee-d3c680007e5c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:27.783292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24baba-2111-4fc7-b7bb-9495cedbb465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:27.783958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24baba-2111-4fc7-b7bb-9495cedbb465 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:27.784301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24baba-2111-4fc7-b7bb-9495cedbb465 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:27.784586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24baba-2111-4fc7-b7bb-9495cedbb465 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:27.796663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d24baba-2111-4fc7-b7bb-9495cedbb465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-0d24baba-2111-4fc7-b7bb-9495cedbb465 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:27.801554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:27.802525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:27.803965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 533/1051] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:27.813736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24baba-2111-4fc7-b7bb-9495cedbb465 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:27.823413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 519/1052] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:27.889427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e79f96-eb0a-4b0e-82ed-c8f8b452dee6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:27.925833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24baba-2111-4fc7-b7bb-9495cedbb465 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:27.956814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:27.963260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65cce0-716b-460e-b0fb-251a5eaf94ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:27.963951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65cce0-716b-460e-b0fb-251a5eaf94ce None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:27.964231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65cce0-716b-460e-b0fb-251a5eaf94ce None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:27.964478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65cce0-716b-460e-b0fb-251a5eaf94ce None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:27.965136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65cce0-716b-460e-b0fb-251a5eaf94ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:27.965434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65cce0-716b-460e-b0fb-251a5eaf94ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:27.967046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65cce0-716b-460e-b0fb-251a5eaf94ce None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:27.975728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 534/1053] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:08:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:27.982292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65cce0-716b-460e-b0fb-251a5eaf94ce None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.010046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e79f96-eb0a-4b0e-82ed-c8f8b452dee6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.030848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:28.046525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9c6d5-93d6-482f-b0b9-f337ce4dbd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.047244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9c6d5-93d6-482f-b0b9-f337ce4dbd89 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.047471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9c6d5-93d6-482f-b0b9-f337ce4dbd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.047820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9c6d5-93d6-482f-b0b9-f337ce4dbd89 None admin] PATH_INFO: `/v3/services/image` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.048388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9c6d5-93d6-482f-b0b9-f337ce4dbd89 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.049741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9c6d5-93d6-482f-b0b9-f337ce4dbd89 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-35e9c6d5-93d6-482f-b0b9-f337ce4dbd89 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:28.058990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:28.060238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 12:08:28.061107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:28.063907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 520/1054] 10.222.0.71 () {56 vars in 1105 bytes} [Wed Sep 21 12:08:27 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:28.066030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9c6d5-93d6-482f-b0b9-f337ce4dbd89 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.098800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65cce0-716b-460e-b0fb-251a5eaf94ce None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.133565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb6ec5-aa18-4590-bc8b-0bbb579e18bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.134192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb6ec5-aa18-4590-bc8b-0bbb579e18bf None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.134478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb6ec5-aa18-4590-bc8b-0bbb579e18bf None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.134749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbb6ec5-aa18-4590-bc8b-0bbb579e18bf None admin] PATH_INFO: `/v3/users/d70ef32c2d8e4734890c16f5faf4dfef` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.148270 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb6ec5-aa18-4590-bc8b-0bbb579e18bf None admin] RBAC: Authorizing `identity:get_user(user_id=d70ef32c2d8e4734890c16f5faf4dfef)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.149725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbb6ec5-aa18-4590-bc8b-0bbb579e18bf None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.151365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 535/1055] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:27 2022] GET /identity/v3/users/d70ef32c2d8e4734890c16f5faf4dfef => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:28.152159 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb6ec5-aa18-4590-bc8b-0bbb579e18bf None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.172742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9c6d5-93d6-482f-b0b9-f337ce4dbd89 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.202996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:28.209275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3b929-cf6f-412b-afc5-3b094183f00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.209882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3b929-cf6f-412b-afc5-3b094183f00d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.210162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3b929-cf6f-412b-afc5-3b094183f00d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.210433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3b929-cf6f-412b-afc5-3b094183f00d None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.211053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3b929-cf6f-412b-afc5-3b094183f00d None admin] RBAC: Adding query filter params (name=image) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:28.211354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3b929-cf6f-412b-afc5-3b094183f00d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.212750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3b929-cf6f-412b-afc5-3b094183f00d None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.217967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 521/1056] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:28.218720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3b929-cf6f-412b-afc5-3b094183f00d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.254579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbb6ec5-aa18-4590-bc8b-0bbb579e18bf None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.267770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:28.287824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fba577-84ca-497d-ad16-16166335243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.288389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fba577-84ca-497d-ad16-16166335243d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.289181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fba577-84ca-497d-ad16-16166335243d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.289454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fba577-84ca-497d-ad16-16166335243d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.299505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fba577-84ca-497d-ad16-16166335243d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-80fba577-84ca-497d-ad16-16166335243d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:28.304874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:28.306032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:28.307075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 536/1057] 10.222.0.71 () {56 vars in 1120 bytes} [Wed Sep 21 12:08:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:28.307962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fba577-84ca-497d-ad16-16166335243d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.322885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3b929-cf6f-412b-afc5-3b094183f00d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.356064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd23f2-c8f6-4d95-818c-c5cb8de6a0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.356670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd23f2-c8f6-4d95-818c-c5cb8de6a0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.356944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd23f2-c8f6-4d95-818c-c5cb8de6a0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.357216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd23f2-c8f6-4d95-818c-c5cb8de6a0c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.408072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fba577-84ca-497d-ad16-16166335243d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.418672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:28.423150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd23f2-c8f6-4d95-818c-c5cb8de6a0c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.424636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd23f2-c8f6-4d95-818c-c5cb8de6a0c7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.443246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852a7f1-fac8-4be4-81f3-1cf27831ab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.443873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852a7f1-fac8-4be4-81f3-1cf27831ab50 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.444152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852a7f1-fac8-4be4-81f3-1cf27831ab50 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.444422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852a7f1-fac8-4be4-81f3-1cf27831ab50 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.445571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852a7f1-fac8-4be4-81f3-1cf27831ab50 None admin] RBAC: Adding query filter params (type=image) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:28.445873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852a7f1-fac8-4be4-81f3-1cf27831ab50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.447230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852a7f1-fac8-4be4-81f3-1cf27831ab50 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.452170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 537/1058] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:28.452966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852a7f1-fac8-4be4-81f3-1cf27831ab50 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.472786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:28.525094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 522/1059] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:08:28.525926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd23f2-c8f6-4d95-818c-c5cb8de6a0c7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.554789 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852a7f1-fac8-4be4-81f3-1cf27831ab50 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.587637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c35e0-5dcd-46da-a860-ac62c4229ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.588401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c35e0-5dcd-46da-a860-ac62c4229ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.588681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c35e0-5dcd-46da-a860-ac62c4229ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.588958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c35e0-5dcd-46da-a860-ac62c4229ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.589614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c35e0-5dcd-46da-a860-ac62c4229ff5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:28.589922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c35e0-5dcd-46da-a860-ac62c4229ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.591536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c35e0-5dcd-46da-a860-ac62c4229ff5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.599321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 538/1060] 10.222.0.71 () {56 vars in 1115 bytes} [Wed Sep 21 12:08:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:28.600149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c35e0-5dcd-46da-a860-ac62c4229ff5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.627283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd23f2-c8f6-4d95-818c-c5cb8de6a0c7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.635216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:28.662912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8c198-2f0a-4e56-81cf-f08ae680de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.663590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8c198-2f0a-4e56-81cf-f08ae680de2b None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.664392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8c198-2f0a-4e56-81cf-f08ae680de2b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.664665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8c198-2f0a-4e56-81cf-f08ae680de2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.665335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8c198-2f0a-4e56-81cf-f08ae680de2b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.666691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8c198-2f0a-4e56-81cf-f08ae680de2b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.686644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 523/1061] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:08:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:28.687473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8c198-2f0a-4e56-81cf-f08ae680de2b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.701153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:28.717317 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c35e0-5dcd-46da-a860-ac62c4229ff5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.789455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8d943-02f1-4cdd-909d-fa5fdc701ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.790518 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8d943-02f1-4cdd-909d-fa5fdc701ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.790898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8d943-02f1-4cdd-909d-fa5fdc701ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.791358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8d943-02f1-4cdd-909d-fa5fdc701ea8 None admin] PATH_INFO: `/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.793286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8d943-02f1-4cdd-909d-fa5fdc701ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3d654f8f2d4b4ea0c238f33b4fce37)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.795835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8d943-02f1-4cdd-909d-fa5fdc701ea8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.798603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 539/1062] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:08:28 2022] GET /identity/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37 => generated 311 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:28.805116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8d943-02f1-4cdd-909d-fa5fdc701ea8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.865556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8c198-2f0a-4e56-81cf-f08ae680de2b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.908372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc172e9-c568-4a49-b18a-7fd6cb15505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.909020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc172e9-c568-4a49-b18a-7fd6cb15505e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.909301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc172e9-c568-4a49-b18a-7fd6cb15505e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.909573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc172e9-c568-4a49-b18a-7fd6cb15505e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.921562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc172e9-c568-4a49-b18a-7fd6cb15505e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.923027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc172e9-c568-4a49-b18a-7fd6cb15505e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.932659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:28.937348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8d943-02f1-4cdd-909d-fa5fdc701ea8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-1dc172e9-c568-4a49-b18a-7fd6cb15505e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:28.941842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:28.942801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:28.944288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:28.944288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:28.944288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:28.944288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 524/1063] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:28.949176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc172e9-c568-4a49-b18a-7fd6cb15505e None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:28.976739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7f5f4-b4a6-4754-8621-8277f4505b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:28.977513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7f5f4-b4a6-4754-8621-8277f4505b6a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:28.978121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7f5f4-b4a6-4754-8621-8277f4505b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:28.978401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce7f5f4-b4a6-4754-8621-8277f4505b6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:28.979282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7f5f4-b4a6-4754-8621-8277f4505b6a None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=2d3d654f8f2d4b4ea0c238f33b4fce37, user.id=d70ef32c2d8e4734890c16f5faf4dfef) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:28.979595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7f5f4-b4a6-4754-8621-8277f4505b6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:28.981087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce7f5f4-b4a6-4754-8621-8277f4505b6a None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:28.986839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 540/1064] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:28 2022] GET /identity/v3/role_assignments?user.id=d70ef32c2d8e4734890c16f5faf4dfef&scope.project.id=2d3d654f8f2d4b4ea0c238f33b4fce37&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:29.010714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:29.063657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc172e9-c568-4a49-b18a-7fd6cb15505e None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:29.102237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55fd0b-acb5-49ca-92b5-6fa2d7a3a79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:29.102861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55fd0b-acb5-49ca-92b5-6fa2d7a3a79a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:29.103141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55fd0b-acb5-49ca-92b5-6fa2d7a3a79a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:29.103411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55fd0b-acb5-49ca-92b5-6fa2d7a3a79a None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:29.104097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55fd0b-acb5-49ca-92b5-6fa2d7a3a79a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:29.104400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55fd0b-acb5-49ca-92b5-6fa2d7a3a79a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:29.105804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55fd0b-acb5-49ca-92b5-6fa2d7a3a79a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:29.124477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 525/1065] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:29.131038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7f5f4-b4a6-4754-8621-8277f4505b6a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:29.238779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce7f5f4-b4a6-4754-8621-8277f4505b6a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:29.274119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506da5c-9c8b-42f5-a432-28b713c935d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:29.274783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506da5c-9c8b-42f5-a432-28b713c935d1 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:29.275023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506da5c-9c8b-42f5-a432-28b713c935d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:29.275277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506da5c-9c8b-42f5-a432-28b713c935d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:29.283971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506da5c-9c8b-42f5-a432-28b713c935d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:29.285421 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506da5c-9c8b-42f5-a432-28b713c935d1 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4506da5c-9c8b-42f5-a432-28b713c935d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:08:29.297255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:29.298229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:08:29.299814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:29.300754 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 541/1066] 10.222.0.71 () {56 vars in 1093 bytes} [Wed Sep 21 12:08:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:29.304709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55fd0b-acb5-49ca-92b5-6fa2d7a3a79a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:29.406885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55fd0b-acb5-49ca-92b5-6fa2d7a3a79a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:29.423692 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:29.443628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f3d82-a1c0-4630-b9e6-7968715fc669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:29.444269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f3d82-a1c0-4630-b9e6-7968715fc669 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:29.444550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f3d82-a1c0-4630-b9e6-7968715fc669 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:29.445402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f3d82-a1c0-4630-b9e6-7968715fc669 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:29.446111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f3d82-a1c0-4630-b9e6-7968715fc669 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:29.446413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f3d82-a1c0-4630-b9e6-7968715fc669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:29.447852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f3d82-a1c0-4630-b9e6-7968715fc669 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:29.465901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 526/1067] 10.222.0.71 () {56 vars in 1133 bytes} [Wed Sep 21 12:08:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:29.471844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506da5c-9c8b-42f5-a432-28b713c935d1 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:29.498374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:29.577612 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506da5c-9c8b-42f5-a432-28b713c935d1 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:29.612744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e81698-b2b9-4766-8bbd-e75e73efbe45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:29.613383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e81698-b2b9-4766-8bbd-e75e73efbe45 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:29.613658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e81698-b2b9-4766-8bbd-e75e73efbe45 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:29.613927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e81698-b2b9-4766-8bbd-e75e73efbe45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:29.625464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e81698-b2b9-4766-8bbd-e75e73efbe45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:29.626900 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e81698-b2b9-4766-8bbd-e75e73efbe45 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-28e81698-b2b9-4766-8bbd-e75e73efbe45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:29.641908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:29.642885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:29.644352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:29.644352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:29.644352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:29.644783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 542/1068] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:29.649212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f3d82-a1c0-4630-b9e6-7968715fc669 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:29.755708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f3d82-a1c0-4630-b9e6-7968715fc669 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:29.795457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80d8e4-5235-4442-8e0d-abc3cecf24e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:29.796083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80d8e4-5235-4442-8e0d-abc3cecf24e2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:29.796355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80d8e4-5235-4442-8e0d-abc3cecf24e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:29.796625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f80d8e4-5235-4442-8e0d-abc3cecf24e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:29.797248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80d8e4-5235-4442-8e0d-abc3cecf24e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:29.797542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80d8e4-5235-4442-8e0d-abc3cecf24e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:29.798886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f80d8e4-5235-4442-8e0d-abc3cecf24e2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:29.810803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 527/1069] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:29.816255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e81698-b2b9-4766-8bbd-e75e73efbe45 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:29.904014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:29.928473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e81698-b2b9-4766-8bbd-e75e73efbe45 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:29.964731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264503f3-d1eb-41be-8476-e1f541e2dcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:29.965362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264503f3-d1eb-41be-8476-e1f541e2dcbc None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:29.965640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264503f3-d1eb-41be-8476-e1f541e2dcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:29.965912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264503f3-d1eb-41be-8476-e1f541e2dcbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:29.973078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264503f3-d1eb-41be-8476-e1f541e2dcbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:29.974537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264503f3-d1eb-41be-8476-e1f541e2dcbc None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-264503f3-d1eb-41be-8476-e1f541e2dcbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:08:29.986000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:29.986874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:29.987783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:08:29.987783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:08:29.987783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:08:29.987783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:08:29.987783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:08:29.987783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:08:29.987783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:29.987783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:29.992084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 543/1070] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:29.992905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80d8e4-5235-4442-8e0d-abc3cecf24e2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:30.029421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:30.095563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f80d8e4-5235-4442-8e0d-abc3cecf24e2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:30.128947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d11ed-b3ea-49c8-99a7-26a28b7a7043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:30.129549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d11ed-b3ea-49c8-99a7-26a28b7a7043 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:30.129830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d11ed-b3ea-49c8-99a7-26a28b7a7043 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:30.130103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d11ed-b3ea-49c8-99a7-26a28b7a7043 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:30.130791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d11ed-b3ea-49c8-99a7-26a28b7a7043 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:30.131096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d11ed-b3ea-49c8-99a7-26a28b7a7043 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:30.133148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d11ed-b3ea-49c8-99a7-26a28b7a7043 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:30.145384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 528/1071] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:08:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:30.151130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264503f3-d1eb-41be-8476-e1f541e2dcbc None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:30.257231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264503f3-d1eb-41be-8476-e1f541e2dcbc None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:30.292092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153d603-d9fa-4f91-95ab-0529b965b228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:30.292735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d603-d9fa-4f91-95ab-0529b965b228 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:30.292955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d603-d9fa-4f91-95ab-0529b965b228 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:30.293196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d603-d9fa-4f91-95ab-0529b965b228 None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/463487449c1d4aada1b5f279c959b0e6/roles/d1f99f1b165f4a919c871bd6722ea445` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:30.295351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153d603-d9fa-4f91-95ab-0529b965b228 None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=463487449c1d4aada1b5f279c959b0e6, role_id=d1f99f1b165f4a919c871bd6722ea445)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:30.296806 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153d603-d9fa-4f91-95ab-0529b965b228 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:30.299873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:30.314741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 544/1072] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:08:30 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/463487449c1d4aada1b5f279c959b0e6/roles/d1f99f1b165f4a919c871bd6722ea445 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:08:30.894516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8403ee-7ef7-42a5-b0c6-5359b9977735 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:30.895049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8403ee-7ef7-42a5-b0c6-5359b9977735 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:30.895049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8403ee-7ef7-42a5-b0c6-5359b9977735 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:30.896584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 529/1073] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:30.902334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaef3fc-c292-464e-81ba-5210c0939643 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:30.902616 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaef3fc-c292-464e-81ba-5210c0939643 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:30.902889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaef3fc-c292-464e-81ba-5210c0939643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:30.921237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-beaef3fc-c292-464e-81ba-5210c0939643 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:30.922417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-beaef3fc-c292-464e-81ba-5210c0939643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:30.995864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 545/1074] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:31.143086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17d56f-9b39-4c23-84e8-33d7fabc10f8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:31.143445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17d56f-9b39-4c23-84e8-33d7fabc10f8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:31.143655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17d56f-9b39-4c23-84e8-33d7fabc10f8 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:31.145062 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 530/1075] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:31.150827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0d05-a526-4c2d-b485-cc060b246828 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:31.151114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0d05-a526-4c2d-b485-cc060b246828 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:31.151386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde0d05-a526-4c2d-b485-cc060b246828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:31.176886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8bde0d05-a526-4c2d-b485-cc060b246828 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:31.178091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8bde0d05-a526-4c2d-b485-cc060b246828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:31.249378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 546/1076] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:31.259980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17d56f-9b39-4c23-84e8-33d7fabc10f8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:31.357917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:31.367061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17d56f-9b39-4c23-84e8-33d7fabc10f8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:31.402135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90f195-0a39-48cb-be6c-6206019ef8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:31.402660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90f195-0a39-48cb-be6c-6206019ef8ab None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:31.402937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90f195-0a39-48cb-be6c-6206019ef8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:31.403229 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90f195-0a39-48cb-be6c-6206019ef8ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:31.472175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90f195-0a39-48cb-be6c-6206019ef8ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:31.473628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90f195-0a39-48cb-be6c-6206019ef8ab None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:31.518569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde0d05-a526-4c2d-b485-cc060b246828 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:31.585584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:31.590568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:31.599710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 531/1077] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:08:31.604651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90f195-0a39-48cb-be6c-6206019ef8ab None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:31.634823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde0d05-a526-4c2d-b485-cc060b246828 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:31.670508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae5501-9155-47bf-816f-471d3a9c8786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:31.671061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae5501-9155-47bf-816f-471d3a9c8786 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:31.671339 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae5501-9155-47bf-816f-471d3a9c8786 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:31.671658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae5501-9155-47bf-816f-471d3a9c8786 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:31.683448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae5501-9155-47bf-816f-471d3a9c8786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-96ae5501-9155-47bf-816f-471d3a9c8786 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:31.688376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:31.689290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:31.690209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 547/1078] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:08:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:31.695926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae5501-9155-47bf-816f-471d3a9c8786 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:31.728836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90f195-0a39-48cb-be6c-6206019ef8ab None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:31.769397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e0661-0549-4b7c-9dcd-fe5b381f6e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:31.770159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e0661-0549-4b7c-9dcd-fe5b381f6e70 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:31.770438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e0661-0549-4b7c-9dcd-fe5b381f6e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:31.770702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e0661-0549-4b7c-9dcd-fe5b381f6e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:31.782099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e0661-0549-4b7c-9dcd-fe5b381f6e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:31.783621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e0661-0549-4b7c-9dcd-fe5b381f6e70 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d03e0661-0549-4b7c-9dcd-fe5b381f6e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:31.798756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:31.799712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:31.800830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:31.800830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:31.800830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:31.800830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 532/1079] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:31.806215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e0661-0549-4b7c-9dcd-fe5b381f6e70 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:31.814674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae5501-9155-47bf-816f-471d3a9c8786 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:31.831602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:31.850631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e5bd3-64c9-4ebb-a790-8426c7efcf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:31.851847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e5bd3-64c9-4ebb-a790-8426c7efcf8e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:31.852125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e5bd3-64c9-4ebb-a790-8426c7efcf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:31.852394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e5bd3-64c9-4ebb-a790-8426c7efcf8e None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:31.853049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e5bd3-64c9-4ebb-a790-8426c7efcf8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:31.853368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e5bd3-64c9-4ebb-a790-8426c7efcf8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:31.854723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e5bd3-64c9-4ebb-a790-8426c7efcf8e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:31.863466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:31.867792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 548/1080] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:08:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:31.869937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e5bd3-64c9-4ebb-a790-8426c7efcf8e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:31.928607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e0661-0549-4b7c-9dcd-fe5b381f6e70 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:31.963682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b938ee-1959-4560-a92e-3833b317cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:31.964340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b938ee-1959-4560-a92e-3833b317cc91 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:31.964597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b938ee-1959-4560-a92e-3833b317cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:31.964925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b938ee-1959-4560-a92e-3833b317cc91 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:31.965493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b938ee-1959-4560-a92e-3833b317cc91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:31.965796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b938ee-1959-4560-a92e-3833b317cc91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:31.967416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b938ee-1959-4560-a92e-3833b317cc91 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:31.987794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 533/1081] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:31.994827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e5bd3-64c9-4ebb-a790-8426c7efcf8e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:32.035416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40261f5-765a-48e1-b27d-4862a156598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:32.036094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40261f5-765a-48e1-b27d-4862a156598d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:32.036376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40261f5-765a-48e1-b27d-4862a156598d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:32.036654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40261f5-765a-48e1-b27d-4862a156598d None admin] PATH_INFO: `/v3/users/d70ef32c2d8e4734890c16f5faf4dfef` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:32.037879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40261f5-765a-48e1-b27d-4862a156598d None admin] RBAC: Authorizing `identity:get_user(user_id=d70ef32c2d8e4734890c16f5faf4dfef)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:32.039853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40261f5-765a-48e1-b27d-4862a156598d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:32.041726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 549/1082] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:31 2022] GET /identity/v3/users/d70ef32c2d8e4734890c16f5faf4dfef => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:32.047603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b938ee-1959-4560-a92e-3833b317cc91 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:32.165234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b938ee-1959-4560-a92e-3833b317cc91 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:32.174175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:32.203006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebed947-bc8f-4029-b718-55902d5d20be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:32.203559 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebed947-bc8f-4029-b718-55902d5d20be None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:32.203867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebed947-bc8f-4029-b718-55902d5d20be None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:32.204137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebed947-bc8f-4029-b718-55902d5d20be None admin] PATH_INFO: `/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:32.205240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebed947-bc8f-4029-b718-55902d5d20be None admin] RBAC: Authorizing `identity:get_project(project_id=2d3d654f8f2d4b4ea0c238f33b4fce37)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:32.206649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebed947-bc8f-4029-b718-55902d5d20be None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:32.208468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 534/1083] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:08:32 2022] GET /identity/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:32.214727 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40261f5-765a-48e1-b27d-4862a156598d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:32.263970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:32.322611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40261f5-765a-48e1-b27d-4862a156598d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:32.357436 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e443161-ef7f-48d6-828b-aa56072d8884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:32.358097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e443161-ef7f-48d6-828b-aa56072d8884 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:32.358377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e443161-ef7f-48d6-828b-aa56072d8884 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:32.358649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e443161-ef7f-48d6-828b-aa56072d8884 None admin] PATH_INFO: `/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37/users/d70ef32c2d8e4734890c16f5faf4dfef/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:32.360597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e443161-ef7f-48d6-828b-aa56072d8884 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d3d654f8f2d4b4ea0c238f33b4fce37, user_id=d70ef32c2d8e4734890c16f5faf4dfef, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:32.362112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e443161-ef7f-48d6-828b-aa56072d8884 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:32.375881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 550/1084] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:08:32 2022] PUT /identity/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37/users/d70ef32c2d8e4734890c16f5faf4dfef/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:08:32.526596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c01792-94d1-48a7-b3b6-8a08cc693190 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:32.526797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c01792-94d1-48a7-b3b6-8a08cc693190 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:32.527050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c01792-94d1-48a7-b3b6-8a08cc693190 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:32.528447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 535/1085] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:32.534240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2d08-fab3-47cf-9555-975b754b7613 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:32.534791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2d08-fab3-47cf-9555-975b754b7613 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:32.534791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a2d08-fab3-47cf-9555-975b754b7613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:32.553507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-013a2d08-fab3-47cf-9555-975b754b7613 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:32.555275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-013a2d08-fab3-47cf-9555-975b754b7613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:32.621977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 551/1086] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:32.897883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c01792-94d1-48a7-b3b6-8a08cc693190 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:32.997377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c01792-94d1-48a7-b3b6-8a08cc693190 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:33.030325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201fb66-522f-4819-9810-dfd2f64fa22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:33.030982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201fb66-522f-4819-9810-dfd2f64fa22f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:33.031260 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201fb66-522f-4819-9810-dfd2f64fa22f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:33.031525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201fb66-522f-4819-9810-dfd2f64fa22f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:33.042510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201fb66-522f-4819-9810-dfd2f64fa22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-0201fb66-522f-4819-9810-dfd2f64fa22f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:33.047272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:33.048403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:33.049374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 536/1087] 10.222.0.71 () {56 vars in 1120 bytes} [Wed Sep 21 12:08:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:33.054580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2d08-fab3-47cf-9555-975b754b7613 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:33.088774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:33.173376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a2d08-fab3-47cf-9555-975b754b7613 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:33.206570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9db016-f086-433e-bbe0-1bbbca569bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:33.207199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9db016-f086-433e-bbe0-1bbbca569bbe None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:33.207476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9db016-f086-433e-bbe0-1bbbca569bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:33.207745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9db016-f086-433e-bbe0-1bbbca569bbe None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:33.208435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9db016-f086-433e-bbe0-1bbbca569bbe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:33.208737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9db016-f086-433e-bbe0-1bbbca569bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:33.210127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9db016-f086-433e-bbe0-1bbbca569bbe None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:33.218642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 552/1088] 10.222.0.71 () {56 vars in 1115 bytes} [Wed Sep 21 12:08:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:33.224557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201fb66-522f-4819-9810-dfd2f64fa22f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:33.260734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:33.328909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201fb66-522f-4819-9810-dfd2f64fa22f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:33.362085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61ce5-c330-46ed-9742-3bab8572eab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:33.362707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61ce5-c330-46ed-9742-3bab8572eab6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:33.363012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61ce5-c330-46ed-9742-3bab8572eab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:33.363280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61ce5-c330-46ed-9742-3bab8572eab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:33.374734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d61ce5-c330-46ed-9742-3bab8572eab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:33.376217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d61ce5-c330-46ed-9742-3bab8572eab6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-89d61ce5-c330-46ed-9742-3bab8572eab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:33.390071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:33.391147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:33.397315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:33.397315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:33.397315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:33.397315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 537/1089] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:33.397597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9db016-f086-433e-bbe0-1bbbca569bbe None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:33.496039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:33.501567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9db016-f086-433e-bbe0-1bbbca569bbe None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:33.538210 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cf7d6-392d-4a7d-8421-ae5575a57dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:33.538895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cf7d6-392d-4a7d-8421-ae5575a57dee None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:33.539073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cf7d6-392d-4a7d-8421-ae5575a57dee None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:33.539354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cf7d6-392d-4a7d-8421-ae5575a57dee None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:33.540012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cf7d6-392d-4a7d-8421-ae5575a57dee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:33.540320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cf7d6-392d-4a7d-8421-ae5575a57dee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:33.541679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90cf7d6-392d-4a7d-8421-ae5575a57dee None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:33.553285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 553/1090] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:33.558948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61ce5-c330-46ed-9742-3bab8572eab6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:33.646543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:33.663291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61ce5-c330-46ed-9742-3bab8572eab6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:33.698474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b933a-eb0c-4d39-9cda-f2a1e72023cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:33.699108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b933a-eb0c-4d39-9cda-f2a1e72023cd None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:33.699387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b933a-eb0c-4d39-9cda-f2a1e72023cd None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:33.699657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b933a-eb0c-4d39-9cda-f2a1e72023cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:33.707269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b933a-eb0c-4d39-9cda-f2a1e72023cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:33.708756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b933a-eb0c-4d39-9cda-f2a1e72023cd None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f14b933a-eb0c-4d39-9cda-f2a1e72023cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:08:33.720668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:33.721751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:33.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 538/1091] 10.222.0.71 () {56 vars in 1093 bytes} [Wed Sep 21 12:08:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:33.728309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cf7d6-392d-4a7d-8421-ae5575a57dee None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:33.840353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cf7d6-392d-4a7d-8421-ae5575a57dee None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:33.876015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3339a-168d-4790-81d0-95e2e65efaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:33.876915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3339a-168d-4790-81d0-95e2e65efaec None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:33.877409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3339a-168d-4790-81d0-95e2e65efaec None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:33.877816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3339a-168d-4790-81d0-95e2e65efaec None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:33.878549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3339a-168d-4790-81d0-95e2e65efaec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:33.878855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3339a-168d-4790-81d0-95e2e65efaec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:33.880305 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3339a-168d-4790-81d0-95e2e65efaec None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:33.898429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 554/1092] 10.222.0.71 () {56 vars in 1133 bytes} [Wed Sep 21 12:08:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:33.904247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b933a-eb0c-4d39-9cda-f2a1e72023cd None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:34.001378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b933a-eb0c-4d39-9cda-f2a1e72023cd None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:34.034321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a28ed2-a9c7-4fdf-a125-e7689e83c824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:34.034849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a28ed2-a9c7-4fdf-a125-e7689e83c824 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:34.035128 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a28ed2-a9c7-4fdf-a125-e7689e83c824 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:34.035397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a28ed2-a9c7-4fdf-a125-e7689e83c824 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:34.050864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:34.102279 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a28ed2-a9c7-4fdf-a125-e7689e83c824 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:34.103715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a28ed2-a9c7-4fdf-a125-e7689e83c824 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:34.200466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 539/1093] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:08:34.206223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3339a-168d-4790-81d0-95e2e65efaec None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:34.224443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:34.307123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3339a-168d-4790-81d0-95e2e65efaec None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:34.340706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e3e03-39cc-4a14-ad61-d3baff5722c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:34.341583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e3e03-39cc-4a14-ad61-d3baff5722c7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:34.341863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e3e03-39cc-4a14-ad61-d3baff5722c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:34.342136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e3e03-39cc-4a14-ad61-d3baff5722c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:34.352592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e3e03-39cc-4a14-ad61-d3baff5722c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:34.354041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e3e03-39cc-4a14-ad61-d3baff5722c7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-564e3e03-39cc-4a14-ad61-d3baff5722c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:08:34.369238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:34.370557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:08:34.371467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:08:34.371467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:08:34.371467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:34.371467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 555/1094] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:34.376146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a28ed2-a9c7-4fdf-a125-e7689e83c824 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:34.421633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:34.478176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a28ed2-a9c7-4fdf-a125-e7689e83c824 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:34.512129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fb753-6395-4bf6-b995-a48962b60a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:34.512732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fb753-6395-4bf6-b995-a48962b60a3a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:34.513011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fb753-6395-4bf6-b995-a48962b60a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:34.513282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fb753-6395-4bf6-b995-a48962b60a3a None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:34.513922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fb753-6395-4bf6-b995-a48962b60a3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:34.514222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fb753-6395-4bf6-b995-a48962b60a3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:34.515583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fb753-6395-4bf6-b995-a48962b60a3a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:34.527302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 540/1095] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:34.533040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e3e03-39cc-4a14-ad61-d3baff5722c7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:34.564316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cde6-0a3e-47d1-bafd-5c7a7a4d5a6a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:34.564608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cde6-0a3e-47d1-bafd-5c7a7a4d5a6a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:34.564883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2cde6-0a3e-47d1-bafd-5c7a7a4d5a6a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:34.566227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 541/1096] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:08:34.571823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5156c94-44e8-48f2-b7f0-7b25e27e3b04 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:34.572116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5156c94-44e8-48f2-b7f0-7b25e27e3b04 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:34.572391 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5156c94-44e8-48f2-b7f0-7b25e27e3b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:34.590108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c5156c94-44e8-48f2-b7f0-7b25e27e3b04 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:34.591557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c5156c94-44e8-48f2-b7f0-7b25e27e3b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:34.601850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:34.636036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e3e03-39cc-4a14-ad61-d3baff5722c7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:34.658492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 542/1097] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:08:34.669426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a38c85-50c0-492d-9832-f328f656a134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:34.670051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a38c85-50c0-492d-9832-f328f656a134 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:34.670328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a38c85-50c0-492d-9832-f328f656a134 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:34.670599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a38c85-50c0-492d-9832-f328f656a134 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:34.677157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a38c85-50c0-492d-9832-f328f656a134 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:34.678589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a38c85-50c0-492d-9832-f328f656a134 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-41a38c85-50c0-492d-9832-f328f656a134 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:08:34.690189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:34.691072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:34.692005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:08:34.692005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:08:34.692005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:08:34.692005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:08:34.692005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:08:34.692005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:08:34.692005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:08:34.692005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:08:34.692005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 556/1098] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:08:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:34.697779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5156c94-44e8-48f2-b7f0-7b25e27e3b04 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:34.732303 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:34.798122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5156c94-44e8-48f2-b7f0-7b25e27e3b04 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:34.830952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14379d9a-2a82-4c10-b193-52ffca9c3a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:34.831486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14379d9a-2a82-4c10-b193-52ffca9c3a79 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:34.831785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14379d9a-2a82-4c10-b193-52ffca9c3a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:34.832064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14379d9a-2a82-4c10-b193-52ffca9c3a79 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:34.832748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14379d9a-2a82-4c10-b193-52ffca9c3a79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:34.833054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14379d9a-2a82-4c10-b193-52ffca9c3a79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:34.834474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14379d9a-2a82-4c10-b193-52ffca9c3a79 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:34.846424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 543/1099] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:08:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:08:34.853263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a38c85-50c0-492d-9832-f328f656a134 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:34.928541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14379d9a-2a82-4c10-b193-52ffca9c3a79 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:34.953772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a38c85-50c0-492d-9832-f328f656a134 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:34.980155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:34.988908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c92e5-0ca0-44a4-81f5-f80633f4bab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:34.989516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c92e5-0ca0-44a4-81f5-f80633f4bab8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:34.989800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c92e5-0ca0-44a4-81f5-f80633f4bab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:34.990074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c92e5-0ca0-44a4-81f5-f80633f4bab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:34.990796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c92e5-0ca0-44a4-81f5-f80633f4bab8 None admin] RBAC: Adding query filter params (role.id=d1f99f1b165f4a919c871bd6722ea445, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=463487449c1d4aada1b5f279c959b0e6) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:34.991104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c92e5-0ca0-44a4-81f5-f80633f4bab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:34.992576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c92e5-0ca0-44a4-81f5-f80633f4bab8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:34.999772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 557/1100] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:34 2022] GET /identity/v3/role_assignments?user.id=463487449c1d4aada1b5f279c959b0e6&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=d1f99f1b165f4a919c871bd6722ea445 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:35.025151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:35.031966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14379d9a-2a82-4c10-b193-52ffca9c3a79 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:35.068765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebfdad-223f-4fbf-b4e9-0e7f66654b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:35.069408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebfdad-223f-4fbf-b4e9-0e7f66654b2b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:35.070184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebfdad-223f-4fbf-b4e9-0e7f66654b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:35.070447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebfdad-223f-4fbf-b4e9-0e7f66654b2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:35.082523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebfdad-223f-4fbf-b4e9-0e7f66654b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-d3ebfdad-223f-4fbf-b4e9-0e7f66654b2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:35.088304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:35.091815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:35.092910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 544/1101] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:08:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:08:35.096337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c92e5-0ca0-44a4-81f5-f80633f4bab8 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:35.205186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c92e5-0ca0-44a4-81f5-f80633f4bab8 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:35.240870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ef985-3b55-4dd8-83c7-0cdfc7d4bb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:35.241506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ef985-3b55-4dd8-83c7-0cdfc7d4bb98 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:35.241784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ef985-3b55-4dd8-83c7-0cdfc7d4bb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:35.242054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ef985-3b55-4dd8-83c7-0cdfc7d4bb98 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:35.242709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ef985-3b55-4dd8-83c7-0cdfc7d4bb98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:35.243011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ef985-3b55-4dd8-83c7-0cdfc7d4bb98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:35.244407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ef985-3b55-4dd8-83c7-0cdfc7d4bb98 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:35.252981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 558/1102] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:08:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:35.259525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebfdad-223f-4fbf-b4e9-0e7f66654b2b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:35.369651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebfdad-223f-4fbf-b4e9-0e7f66654b2b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:35.404135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ac816-dc53-40eb-8600-99b1b5a0a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:35.404769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ac816-dc53-40eb-8600-99b1b5a0a0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:35.405047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ac816-dc53-40eb-8600-99b1b5a0a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:35.405319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ac816-dc53-40eb-8600-99b1b5a0a0c5 None admin] PATH_INFO: `/v3/users/d70ef32c2d8e4734890c16f5faf4dfef` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:35.406439 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ac816-dc53-40eb-8600-99b1b5a0a0c5 None admin] RBAC: Authorizing `identity:get_user(user_id=d70ef32c2d8e4734890c16f5faf4dfef)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:35.407891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8ac816-dc53-40eb-8600-99b1b5a0a0c5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:35.409474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 545/1103] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:35 2022] GET /identity/v3/users/d70ef32c2d8e4734890c16f5faf4dfef => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:35.415742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ef985-3b55-4dd8-83c7-0cdfc7d4bb98 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:35.473177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:35.519718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ef985-3b55-4dd8-83c7-0cdfc7d4bb98 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:35.555095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfff45-acce-4b9c-9eda-223a83fcc9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:35.555703 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfff45-acce-4b9c-9eda-223a83fcc9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:35.556010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfff45-acce-4b9c-9eda-223a83fcc9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:35.556282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfff45-acce-4b9c-9eda-223a83fcc9f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:35.625861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfff45-acce-4b9c-9eda-223a83fcc9f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:35.627243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfff45-acce-4b9c-9eda-223a83fcc9f5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:35.682618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:35.730257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 559/1104] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:08:35.736283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ac816-dc53-40eb-8600-99b1b5a0a0c5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:35.765797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:35.846930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ac816-dc53-40eb-8600-99b1b5a0a0c5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:35.904057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95f0e4-cea1-40ae-9700-070bac15450e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:35.904650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f0e4-cea1-40ae-9700-070bac15450e None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:35.904923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f0e4-cea1-40ae-9700-070bac15450e None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:35.905212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95f0e4-cea1-40ae-9700-070bac15450e None admin] PATH_INFO: `/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:35.906407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95f0e4-cea1-40ae-9700-070bac15450e None admin] RBAC: Authorizing `identity:get_project(project_id=2d3d654f8f2d4b4ea0c238f33b4fce37)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:35.907775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95f0e4-cea1-40ae-9700-070bac15450e None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:35.909409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 546/1105] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:08:35 2022] GET /identity/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:35.916326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfff45-acce-4b9c-9eda-223a83fcc9f5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:36.012014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfff45-acce-4b9c-9eda-223a83fcc9f5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:36.045688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1863259-1abe-4134-9bfb-2edfe772c4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:36.046277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1863259-1abe-4134-9bfb-2edfe772c4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:36.046554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1863259-1abe-4134-9bfb-2edfe772c4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:36.046822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1863259-1abe-4134-9bfb-2edfe772c4d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:36.047523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1863259-1abe-4134-9bfb-2edfe772c4d8 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=2d3d654f8f2d4b4ea0c238f33b4fce37, user.id=d70ef32c2d8e4734890c16f5faf4dfef) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:36.047863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1863259-1abe-4134-9bfb-2edfe772c4d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:36.049265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1863259-1abe-4134-9bfb-2edfe772c4d8 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:36.054822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 560/1106] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:35 2022] GET /identity/v3/role_assignments?user.id=d70ef32c2d8e4734890c16f5faf4dfef&scope.project.id=2d3d654f8f2d4b4ea0c238f33b4fce37&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:38.241397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c475de8-843e-45b0-8595-c140bb734487 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:38.243682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c475de8-843e-45b0-8595-c140bb734487 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:38.243682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c475de8-843e-45b0-8595-c140bb734487 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:38.243682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 547/1107] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:38.248863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab70680-93ae-41b5-840e-28ed5a56d880 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:38.249274 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab70680-93ae-41b5-840e-28ed5a56d880 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:38.249413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab70680-93ae-41b5-840e-28ed5a56d880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:38.268427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4ab70680-93ae-41b5-840e-28ed5a56d880 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:38.269615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4ab70680-93ae-41b5-840e-28ed5a56d880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:38.283132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:38.337669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 561/1108] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:38.605794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c475de8-843e-45b0-8595-c140bb734487 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:38.696546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:38.706383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c475de8-843e-45b0-8595-c140bb734487 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:38.738828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1439a-7113-4fe7-b5bd-1223baec977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:38.739441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1439a-7113-4fe7-b5bd-1223baec977a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:38.739717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1439a-7113-4fe7-b5bd-1223baec977a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:38.740014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1439a-7113-4fe7-b5bd-1223baec977a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:38.741127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1439a-7113-4fe7-b5bd-1223baec977a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:38.743101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1439a-7113-4fe7-b5bd-1223baec977a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:38.745008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 548/1109] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:38.750586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab70680-93ae-41b5-840e-28ed5a56d880 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:38.847442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab70680-93ae-41b5-840e-28ed5a56d880 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:38.885811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58dd0f-0172-462b-b138-61be4844b7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:38.886342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58dd0f-0172-462b-b138-61be4844b7d9 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:38.886623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58dd0f-0172-462b-b138-61be4844b7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:38.886898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58dd0f-0172-462b-b138-61be4844b7d9 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:38.887556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58dd0f-0172-462b-b138-61be4844b7d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:38.889032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58dd0f-0172-462b-b138-61be4844b7d9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:38.910442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 562/1110] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:08:38 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:41.093137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2a630-cad9-4206-bcb1-6fadff563f16 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:41.093670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2a630-cad9-4206-bcb1-6fadff563f16 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:41.093670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2a630-cad9-4206-bcb1-6fadff563f16 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:41.094992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 549/1111] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:41.100120 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebc222-5829-4565-8685-e231c1d41758 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:41.101085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebc222-5829-4565-8685-e231c1d41758 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:41.101186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebc222-5829-4565-8685-e231c1d41758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:41.108386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:41.120364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e6ebc222-5829-4565-8685-e231c1d41758 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:41.121554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e6ebc222-5829-4565-8685-e231c1d41758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:41.185659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 563/1112] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:41.452231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2a630-cad9-4206-bcb1-6fadff563f16 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:41.550695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2a630-cad9-4206-bcb1-6fadff563f16 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:41.582486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b35f53-a819-41c3-a01c-5ef4efa814c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:41.583102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b35f53-a819-41c3-a01c-5ef4efa814c8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:41.583380 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b35f53-a819-41c3-a01c-5ef4efa814c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:41.583650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b35f53-a819-41c3-a01c-5ef4efa814c8 None admin] PATH_INFO: `/v3/roles/23d9f4769fd6424bae83090a691c7f94` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:41.585605 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b35f53-a819-41c3-a01c-5ef4efa814c8 None admin] RBAC: Authorizing `identity:get_role(role_id=23d9f4769fd6424bae83090a691c7f94)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:41.587000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b35f53-a819-41c3-a01c-5ef4efa814c8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:41.588348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 550/1113] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:41 2022] GET /identity/v3/roles/23d9f4769fd6424bae83090a691c7f94 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:41.594400 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebc222-5829-4565-8685-e231c1d41758 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:41.695912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebc222-5829-4565-8685-e231c1d41758 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:41.731378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63ce3a-8fe3-40c7-b32e-b75f9557be54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:41.732233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63ce3a-8fe3-40c7-b32e-b75f9557be54 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:41.732726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63ce3a-8fe3-40c7-b32e-b75f9557be54 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:41.733214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63ce3a-8fe3-40c7-b32e-b75f9557be54 None admin] PATH_INFO: `/v3/users/33a812a2f0644ea9b22571d950ef8d33` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:41.735160 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:41.747992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63ce3a-8fe3-40c7-b32e-b75f9557be54 None admin] RBAC: Authorizing `identity:get_user(user_id=33a812a2f0644ea9b22571d950ef8d33)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:41.749452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63ce3a-8fe3-40c7-b32e-b75f9557be54 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:41.751106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 564/1114] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:41 2022] GET /identity/v3/users/33a812a2f0644ea9b22571d950ef8d33 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:41.757388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b35f53-a819-41c3-a01c-5ef4efa814c8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:41.785831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:41.855938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b35f53-a819-41c3-a01c-5ef4efa814c8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:41.888492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef45f83-fff1-4f7f-b24b-408def176e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:41.889100 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef45f83-fff1-4f7f-b24b-408def176e78 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:41.889383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef45f83-fff1-4f7f-b24b-408def176e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:41.889658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef45f83-fff1-4f7f-b24b-408def176e78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:41.956532 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef45f83-fff1-4f7f-b24b-408def176e78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:41.957966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef45f83-fff1-4f7f-b24b-408def176e78 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:41.984547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:42.053441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 551/1115] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:08:42.059918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63ce3a-8fe3-40c7-b32e-b75f9557be54 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:42.155983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63ce3a-8fe3-40c7-b32e-b75f9557be54 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:42.188643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bef648-fd30-4226-9b0f-28071cd3e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:42.189260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef648-fd30-4226-9b0f-28071cd3e951 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:42.189533 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef648-fd30-4226-9b0f-28071cd3e951 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:42.189798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bef648-fd30-4226-9b0f-28071cd3e951 None admin] PATH_INFO: `/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:42.190899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bef648-fd30-4226-9b0f-28071cd3e951 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3d654f8f2d4b4ea0c238f33b4fce37)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:42.192358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bef648-fd30-4226-9b0f-28071cd3e951 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:42.193970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 565/1116] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:08:42 2022] GET /identity/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:42.200769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef45f83-fff1-4f7f-b24b-408def176e78 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:42.296539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef45f83-fff1-4f7f-b24b-408def176e78 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:42.323184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:42.330946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d056164-87d4-41ee-bc42-fd51384ae3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:42.331547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d056164-87d4-41ee-bc42-fd51384ae3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:42.331860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d056164-87d4-41ee-bc42-fd51384ae3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:42.332129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d056164-87d4-41ee-bc42-fd51384ae3e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:42.332845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d056164-87d4-41ee-bc42-fd51384ae3e0 None admin] RBAC: Adding query filter params (role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=2d3d654f8f2d4b4ea0c238f33b4fce37, user.id=33a812a2f0644ea9b22571d950ef8d33) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:42.333158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d056164-87d4-41ee-bc42-fd51384ae3e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:42.334577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d056164-87d4-41ee-bc42-fd51384ae3e0 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:42.339858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 552/1117] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:42 2022] GET /identity/v3/role_assignments?user.id=33a812a2f0644ea9b22571d950ef8d33&scope.project.id=2d3d654f8f2d4b4ea0c238f33b4fce37&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:44.587184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8c4f6-f6e3-4ea1-a595-4d3108ffe579 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:44.587721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8c4f6-f6e3-4ea1-a595-4d3108ffe579 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:44.587721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8c4f6-f6e3-4ea1-a595-4d3108ffe579 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:44.589216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 566/1118] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:44.594564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1eae41-a753-4450-b2ea-61ead01477bf None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:44.594960 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1eae41-a753-4450-b2ea-61ead01477bf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:44.595108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1eae41-a753-4450-b2ea-61ead01477bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:44.612801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bf1eae41-a753-4450-b2ea-61ead01477bf None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:44.613978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bf1eae41-a753-4450-b2ea-61ead01477bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:44.679525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 553/1119] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:44.948020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8c4f6-f6e3-4ea1-a595-4d3108ffe579 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:45.007976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:45.051608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8c4f6-f6e3-4ea1-a595-4d3108ffe579 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:45.087129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b97c7-6a4a-4fb2-844f-14ffa123b80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:45.087742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b97c7-6a4a-4fb2-844f-14ffa123b80c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:45.088044 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b97c7-6a4a-4fb2-844f-14ffa123b80c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:45.088310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b97c7-6a4a-4fb2-844f-14ffa123b80c None admin] PATH_INFO: `/v3/roles/23d9f4769fd6424bae83090a691c7f94` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:45.089673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b97c7-6a4a-4fb2-844f-14ffa123b80c None admin] RBAC: Authorizing `identity:get_role(role_id=23d9f4769fd6424bae83090a691c7f94)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:45.091268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b97c7-6a4a-4fb2-844f-14ffa123b80c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:45.092600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 567/1120] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:44 2022] GET /identity/v3/roles/23d9f4769fd6424bae83090a691c7f94 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:45.098570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1eae41-a753-4450-b2ea-61ead01477bf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:45.193571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1eae41-a753-4450-b2ea-61ead01477bf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:45.215821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:45.227623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca74f15-522e-438c-9d28-008c3cefc6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:45.228267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca74f15-522e-438c-9d28-008c3cefc6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:45.228545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca74f15-522e-438c-9d28-008c3cefc6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:45.228813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca74f15-522e-438c-9d28-008c3cefc6f3 None admin] PATH_INFO: `/v3/users/33a812a2f0644ea9b22571d950ef8d33` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:45.229916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca74f15-522e-438c-9d28-008c3cefc6f3 None admin] RBAC: Authorizing `identity:get_user(user_id=33a812a2f0644ea9b22571d950ef8d33)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:45.231343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca74f15-522e-438c-9d28-008c3cefc6f3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:45.232933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 554/1121] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:45 2022] GET /identity/v3/users/33a812a2f0644ea9b22571d950ef8d33 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:45.238849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b97c7-6a4a-4fb2-844f-14ffa123b80c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:45.339617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b97c7-6a4a-4fb2-844f-14ffa123b80c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:45.366514 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:45.374913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301e866-1506-4605-acd7-474ab7c26a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:45.375538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301e866-1506-4605-acd7-474ab7c26a4d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:45.375847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301e866-1506-4605-acd7-474ab7c26a4d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:45.376122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301e866-1506-4605-acd7-474ab7c26a4d None admin] PATH_INFO: `/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:45.377228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301e866-1506-4605-acd7-474ab7c26a4d None admin] RBAC: Authorizing `identity:get_project(project_id=2d3d654f8f2d4b4ea0c238f33b4fce37)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:45.378656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301e866-1506-4605-acd7-474ab7c26a4d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:45.380313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 568/1122] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:08:45 2022] GET /identity/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:45.386459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca74f15-522e-438c-9d28-008c3cefc6f3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:45.484932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca74f15-522e-438c-9d28-008c3cefc6f3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:45.518221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75190878-e6fe-4b50-8072-b57216f5d9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:45.518873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75190878-e6fe-4b50-8072-b57216f5d9cc None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:45.519154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75190878-e6fe-4b50-8072-b57216f5d9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:45.519430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75190878-e6fe-4b50-8072-b57216f5d9cc None admin] PATH_INFO: `/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37/users/33a812a2f0644ea9b22571d950ef8d33/roles/23d9f4769fd6424bae83090a691c7f94` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:45.521586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75190878-e6fe-4b50-8072-b57216f5d9cc None admin] RBAC: Authorizing `identity:create_grant(project_id=2d3d654f8f2d4b4ea0c238f33b4fce37, user_id=33a812a2f0644ea9b22571d950ef8d33, role_id=23d9f4769fd6424bae83090a691c7f94)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:45.523927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75190878-e6fe-4b50-8072-b57216f5d9cc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:45.536466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 555/1123] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:08:45 2022] PUT /identity/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37/users/33a812a2f0644ea9b22571d950ef8d33/roles/23d9f4769fd6424bae83090a691c7f94 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:08:47.674439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29bb41-2173-4f4e-ac89-8f388be41f3c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:47.674980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29bb41-2173-4f4e-ac89-8f388be41f3c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:47.674980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29bb41-2173-4f4e-ac89-8f388be41f3c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:47.676349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 569/1124] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:47.686012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34900b67-4401-49cf-ba48-0af0cbfc8221 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:47.686528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34900b67-4401-49cf-ba48-0af0cbfc8221 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:47.686528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34900b67-4401-49cf-ba48-0af0cbfc8221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:47.705773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-34900b67-4401-49cf-ba48-0af0cbfc8221 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:47.706970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-34900b67-4401-49cf-ba48-0af0cbfc8221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:47.754832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:47.777980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 556/1125] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:48.055641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29bb41-2173-4f4e-ac89-8f388be41f3c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:48.157186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29bb41-2173-4f4e-ac89-8f388be41f3c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:48.190105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46faabd-fe39-4f3f-99c2-db95b483a876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:48.190721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46faabd-fe39-4f3f-99c2-db95b483a876 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:48.190997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46faabd-fe39-4f3f-99c2-db95b483a876 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:48.191268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46faabd-fe39-4f3f-99c2-db95b483a876 None admin] PATH_INFO: `/v3/roles/23d9f4769fd6424bae83090a691c7f94` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:48.192648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46faabd-fe39-4f3f-99c2-db95b483a876 None admin] RBAC: Authorizing `identity:get_role(role_id=23d9f4769fd6424bae83090a691c7f94)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:48.194597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46faabd-fe39-4f3f-99c2-db95b483a876 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:48.195932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 570/1126] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:48 2022] GET /identity/v3/roles/23d9f4769fd6424bae83090a691c7f94 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:48.201750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34900b67-4401-49cf-ba48-0af0cbfc8221 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:48.299503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34900b67-4401-49cf-ba48-0af0cbfc8221 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:48.333379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102fbe3-9db5-4b6b-8c92-a5481ced45de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:48.334002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102fbe3-9db5-4b6b-8c92-a5481ced45de None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:48.334282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102fbe3-9db5-4b6b-8c92-a5481ced45de None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:48.334581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102fbe3-9db5-4b6b-8c92-a5481ced45de None admin] PATH_INFO: `/v3/users/33a812a2f0644ea9b22571d950ef8d33` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:48.335703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102fbe3-9db5-4b6b-8c92-a5481ced45de None admin] RBAC: Authorizing `identity:get_user(user_id=33a812a2f0644ea9b22571d950ef8d33)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:48.337177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102fbe3-9db5-4b6b-8c92-a5481ced45de None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:48.338733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 557/1127] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:48 2022] GET /identity/v3/users/33a812a2f0644ea9b22571d950ef8d33 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:48.344955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46faabd-fe39-4f3f-99c2-db95b483a876 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:48.406208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:48.443769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46faabd-fe39-4f3f-99c2-db95b483a876 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:48.477358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c06e-53cf-49fd-90e6-e561e3c7dc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:48.477955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c06e-53cf-49fd-90e6-e561e3c7dc04 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:48.478235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c06e-53cf-49fd-90e6-e561e3c7dc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:48.478506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3c06e-53cf-49fd-90e6-e561e3c7dc04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:48.552254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3c06e-53cf-49fd-90e6-e561e3c7dc04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:48.553862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3c06e-53cf-49fd-90e6-e561e3c7dc04 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:48.612738 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:48.650979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 571/1128] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:08:48.657043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102fbe3-9db5-4b6b-8c92-a5481ced45de None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:48.704578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:48.760522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102fbe3-9db5-4b6b-8c92-a5481ced45de None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:48.793767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c3409-b5ee-495c-960a-c0c3d05191b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:48.794389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c3409-b5ee-495c-960a-c0c3d05191b5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:48.794669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c3409-b5ee-495c-960a-c0c3d05191b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:48.794942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c3409-b5ee-495c-960a-c0c3d05191b5 None admin] PATH_INFO: `/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:48.796142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c3409-b5ee-495c-960a-c0c3d05191b5 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3d654f8f2d4b4ea0c238f33b4fce37)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:48.797567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c3409-b5ee-495c-960a-c0c3d05191b5 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:48.799195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 558/1129] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:08:48 2022] GET /identity/v3/projects/2d3d654f8f2d4b4ea0c238f33b4fce37 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:48.805808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c06e-53cf-49fd-90e6-e561e3c7dc04 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:48.908292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3c06e-53cf-49fd-90e6-e561e3c7dc04 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:48.942406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470eb10a-cd7d-4438-9fac-11b60e77c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:48.943005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470eb10a-cd7d-4438-9fac-11b60e77c760 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:48.943285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470eb10a-cd7d-4438-9fac-11b60e77c760 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:48.943557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470eb10a-cd7d-4438-9fac-11b60e77c760 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:48.944935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470eb10a-cd7d-4438-9fac-11b60e77c760 None admin] RBAC: Adding query filter params (role.id=23d9f4769fd6424bae83090a691c7f94, scope.project.id=2d3d654f8f2d4b4ea0c238f33b4fce37, user.id=33a812a2f0644ea9b22571d950ef8d33) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:48.945248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470eb10a-cd7d-4438-9fac-11b60e77c760 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:48.946674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470eb10a-cd7d-4438-9fac-11b60e77c760 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:48.952329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 572/1130] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:48 2022] GET /identity/v3/role_assignments?user.id=33a812a2f0644ea9b22571d950ef8d33&scope.project.id=2d3d654f8f2d4b4ea0c238f33b4fce37&role.id=23d9f4769fd6424bae83090a691c7f94 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:51.104771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a26f49-7540-49cb-a86a-ce30062ead2c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:51.105309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a26f49-7540-49cb-a86a-ce30062ead2c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:51.105309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a26f49-7540-49cb-a86a-ce30062ead2c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:51.106657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 559/1131] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:51.111749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990d607-cd9b-42ab-8301-0b0de16a1cfa None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:51.112142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990d607-cd9b-42ab-8301-0b0de16a1cfa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:51.112322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990d607-cd9b-42ab-8301-0b0de16a1cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:51.131253 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2990d607-cd9b-42ab-8301-0b0de16a1cfa None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:51.133032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2990d607-cd9b-42ab-8301-0b0de16a1cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:51.155318 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:51.198717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 573/1132] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:51.467198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a26f49-7540-49cb-a86a-ce30062ead2c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:51.577762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a26f49-7540-49cb-a86a-ce30062ead2c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:51.601299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:51.619451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be2f8f-f79d-4c4e-976c-5c0d162803b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:51.620112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be2f8f-f79d-4c4e-976c-5c0d162803b3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:51.620409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be2f8f-f79d-4c4e-976c-5c0d162803b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:51.620673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be2f8f-f79d-4c4e-976c-5c0d162803b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:51.621760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be2f8f-f79d-4c4e-976c-5c0d162803b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:51.623175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be2f8f-f79d-4c4e-976c-5c0d162803b3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:51.624783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 560/1133] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:51.630671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990d607-cd9b-42ab-8301-0b0de16a1cfa None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:51.729820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990d607-cd9b-42ab-8301-0b0de16a1cfa None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:51.763565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43ed58-8f7a-4edd-8baf-ca09bf42e9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:51.764170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43ed58-8f7a-4edd-8baf-ca09bf42e9e9 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:51.764449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43ed58-8f7a-4edd-8baf-ca09bf42e9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:51.764721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43ed58-8f7a-4edd-8baf-ca09bf42e9e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:51.765378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43ed58-8f7a-4edd-8baf-ca09bf42e9e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:51.766815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43ed58-8f7a-4edd-8baf-ca09bf42e9e9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:51.794020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 574/1134] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:08:51 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:53.979300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87995a0-8aa6-4ebd-b638-ecee93a78de1 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:53.979924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87995a0-8aa6-4ebd-b638-ecee93a78de1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:53.979924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87995a0-8aa6-4ebd-b638-ecee93a78de1 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:53.981213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 561/1135] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:53.986432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed674472-d43f-4a81-97a3-335c5b62a8ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:53.986861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed674472-d43f-4a81-97a3-335c5b62a8ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:53.986983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed674472-d43f-4a81-97a3-335c5b62a8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:54.004800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ed674472-d43f-4a81-97a3-335c5b62a8ad None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:54.005986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ed674472-d43f-4a81-97a3-335c5b62a8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:54.009602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:54.073626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 575/1136] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:54.356768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87995a0-8aa6-4ebd-b638-ecee93a78de1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:54.455869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87995a0-8aa6-4ebd-b638-ecee93a78de1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:54.488181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982fa86-5631-46b0-b781-6717e31fd2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:54.488789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982fa86-5631-46b0-b781-6717e31fd2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:54.489061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982fa86-5631-46b0-b781-6717e31fd2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:54.489323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982fa86-5631-46b0-b781-6717e31fd2e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:54.490402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982fa86-5631-46b0-b781-6717e31fd2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:54.492545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982fa86-5631-46b0-b781-6717e31fd2e6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:54.494075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 562/1137] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:08:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:54.499854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed674472-d43f-4a81-97a3-335c5b62a8ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:54.602013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed674472-d43f-4a81-97a3-335c5b62a8ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:54.635955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dcbe6-295c-4285-b69c-f5b3e7f64765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:54.636481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dcbe6-295c-4285-b69c-f5b3e7f64765 None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:54.636762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dcbe6-295c-4285-b69c-f5b3e7f64765 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:54.637076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dcbe6-295c-4285-b69c-f5b3e7f64765 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:54.638023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dcbe6-295c-4285-b69c-f5b3e7f64765 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:54.639460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dcbe6-295c-4285-b69c-f5b3e7f64765 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:54.650278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:54.661481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 576/1138] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:08:54 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:08:56.835899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e29d7-21f1-483e-b5b3-4b1fd56518a3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:56.836430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e29d7-21f1-483e-b5b3-4b1fd56518a3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:56.836524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e29d7-21f1-483e-b5b3-4b1fd56518a3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:56.837831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 563/1139] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:08:56.843450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e039eb2-66ae-4db0-aacb-fed31f5919a0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:56.843869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e039eb2-66ae-4db0-aacb-fed31f5919a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:56.844043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e039eb2-66ae-4db0-aacb-fed31f5919a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:56.861954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7e039eb2-66ae-4db0-aacb-fed31f5919a0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:08:56.863145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7e039eb2-66ae-4db0-aacb-fed31f5919a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:08:56.928971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 577/1140] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:08:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:08:57.200012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e29d7-21f1-483e-b5b3-4b1fd56518a3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:57.253359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:57.299901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e29d7-21f1-483e-b5b3-4b1fd56518a3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:57.334682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae01725-8801-4685-a093-17239cb1c21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:57.335311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae01725-8801-4685-a093-17239cb1c21d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:57.335592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae01725-8801-4685-a093-17239cb1c21d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:57.335902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae01725-8801-4685-a093-17239cb1c21d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:57.346295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae01725-8801-4685-a093-17239cb1c21d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-bae01725-8801-4685-a093-17239cb1c21d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:08:57.351117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:08:57.352159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:08:57.353092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 564/1141] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:08:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:08:57.358322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e039eb2-66ae-4db0-aacb-fed31f5919a0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:57.456257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e039eb2-66ae-4db0-aacb-fed31f5919a0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:57.481532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:57.490085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441a6f2-a77d-4576-b13a-f68fca531ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:57.491234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441a6f2-a77d-4576-b13a-f68fca531ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:57.491510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441a6f2-a77d-4576-b13a-f68fca531ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:57.491802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441a6f2-a77d-4576-b13a-f68fca531ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:57.492464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441a6f2-a77d-4576-b13a-f68fca531ad5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:57.492767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441a6f2-a77d-4576-b13a-f68fca531ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:57.494128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441a6f2-a77d-4576-b13a-f68fca531ad5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:57.502135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 578/1142] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:08:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:57.508291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae01725-8801-4685-a093-17239cb1c21d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:57.606789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae01725-8801-4685-a093-17239cb1c21d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:57.620963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:57.641991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c99d5-faef-45e1-bb8d-34f872043993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:57.642620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c99d5-faef-45e1-bb8d-34f872043993 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:57.642898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c99d5-faef-45e1-bb8d-34f872043993 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:57.643192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c99d5-faef-45e1-bb8d-34f872043993 None admin] PATH_INFO: `/v3/users/9f460e459de440ae86e8e754ba4941b1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:57.656023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c99d5-faef-45e1-bb8d-34f872043993 None admin] RBAC: Authorizing `identity:get_user(user_id=9f460e459de440ae86e8e754ba4941b1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:57.657464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c99d5-faef-45e1-bb8d-34f872043993 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:57.659064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 565/1143] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:08:57 2022] GET /identity/v3/users/9f460e459de440ae86e8e754ba4941b1 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:57.665464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441a6f2-a77d-4576-b13a-f68fca531ad5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:57.766183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441a6f2-a77d-4576-b13a-f68fca531ad5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:57.799678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec6dd6-c58e-4808-8c61-7c30d1d8f07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:57.800728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec6dd6-c58e-4808-8c61-7c30d1d8f07c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:57.800728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec6dd6-c58e-4808-8c61-7c30d1d8f07c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:57.800951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec6dd6-c58e-4808-8c61-7c30d1d8f07c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:57.855734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:57.868988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec6dd6-c58e-4808-8c61-7c30d1d8f07c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:57.870451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec6dd6-c58e-4808-8c61-7c30d1d8f07c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:57.965164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 579/1144] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:08:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:08:57.971370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c99d5-faef-45e1-bb8d-34f872043993 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:58.072686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c99d5-faef-45e1-bb8d-34f872043993 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:58.106335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bb70d-e277-48d0-acb2-fa10a2f3f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:58.106955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bb70d-e277-48d0-acb2-fa10a2f3f777 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:58.107236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bb70d-e277-48d0-acb2-fa10a2f3f777 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:58.107510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bb70d-e277-48d0-acb2-fa10a2f3f777 None admin] PATH_INFO: `/v3/projects/ae3a7bdb06c94e1688734f0f8f1f65be` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:58.108642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bb70d-e277-48d0-acb2-fa10a2f3f777 None admin] RBAC: Authorizing `identity:get_project(project_id=ae3a7bdb06c94e1688734f0f8f1f65be)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:58.110068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bb70d-e277-48d0-acb2-fa10a2f3f777 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:58.111688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 566/1145] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:08:57 2022] GET /identity/v3/projects/ae3a7bdb06c94e1688734f0f8f1f65be => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:08:58.117882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec6dd6-c58e-4808-8c61-7c30d1d8f07c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:08:58.210557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:08:58.223400 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec6dd6-c58e-4808-8c61-7c30d1d8f07c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:08:58.256856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cc4d9-ea49-4d06-8f7d-9e8c54b5bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:08:58.257457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cc4d9-ea49-4d06-8f7d-9e8c54b5bc6f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:08:58.257737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cc4d9-ea49-4d06-8f7d-9e8c54b5bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:08:58.258011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cc4d9-ea49-4d06-8f7d-9e8c54b5bc6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:08:58.258728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cc4d9-ea49-4d06-8f7d-9e8c54b5bc6f None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=ae3a7bdb06c94e1688734f0f8f1f65be, user.id=9f460e459de440ae86e8e754ba4941b1) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:08:58.259037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cc4d9-ea49-4d06-8f7d-9e8c54b5bc6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:08:58.260486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cc4d9-ea49-4d06-8f7d-9e8c54b5bc6f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:08:58.265686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 580/1146] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:08:58 2022] GET /identity/v3/role_assignments?user.id=9f460e459de440ae86e8e754ba4941b1&scope.project.id=ae3a7bdb06c94e1688734f0f8f1f65be&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:00.378165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b5edc-f32e-4f87-9f49-c44cd111d85c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:00.378720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b5edc-f32e-4f87-9f49-c44cd111d85c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:00.378720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b5edc-f32e-4f87-9f49-c44cd111d85c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:00.380165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 567/1147] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:09:00.385225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fb3a7-5190-45a7-9921-69aa0588848c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:00.385640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fb3a7-5190-45a7-9921-69aa0588848c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:00.385778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fb3a7-5190-45a7-9921-69aa0588848c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:00.404373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d84fb3a7-5190-45a7-9921-69aa0588848c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:09:00.405568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d84fb3a7-5190-45a7-9921-69aa0588848c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:09:00.471235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 581/1148] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:09:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:09:00.741202 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b5edc-f32e-4f87-9f49-c44cd111d85c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:00.781367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:00.840245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b5edc-f32e-4f87-9f49-c44cd111d85c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:00.873293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86d156-87db-4168-bc50-e016d249bed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:00.873901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86d156-87db-4168-bc50-e016d249bed5 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:00.874175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86d156-87db-4168-bc50-e016d249bed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:00.874439 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86d156-87db-4168-bc50-e016d249bed5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:00.884594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea86d156-87db-4168-bc50-e016d249bed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ea86d156-87db-4168-bc50-e016d249bed5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:09:00.889337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:09:00.890479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:09:00.891618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 568/1149] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:09:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:09:00.896733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fb3a7-5190-45a7-9921-69aa0588848c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:00.974759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:00.993125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fb3a7-5190-45a7-9921-69aa0588848c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:01.026503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a5536-1f7a-4796-aeb9-f05978ca66f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:01.027098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a5536-1f7a-4796-aeb9-f05978ca66f9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:01.027375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a5536-1f7a-4796-aeb9-f05978ca66f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:01.027643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220a5536-1f7a-4796-aeb9-f05978ca66f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:01.028321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a5536-1f7a-4796-aeb9-f05978ca66f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:09:01.028620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a5536-1f7a-4796-aeb9-f05978ca66f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:01.029972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220a5536-1f7a-4796-aeb9-f05978ca66f9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:01.037690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 582/1150] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:09:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:01.043500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86d156-87db-4168-bc50-e016d249bed5 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:01.137417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:01.141321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86d156-87db-4168-bc50-e016d249bed5 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:01.174673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01cbf2-9bd0-4147-8bb1-ee328e7ebea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:01.175293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01cbf2-9bd0-4147-8bb1-ee328e7ebea2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:01.175573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01cbf2-9bd0-4147-8bb1-ee328e7ebea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:01.175875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01cbf2-9bd0-4147-8bb1-ee328e7ebea2 None admin] PATH_INFO: `/v3/users/9f460e459de440ae86e8e754ba4941b1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:01.176986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01cbf2-9bd0-4147-8bb1-ee328e7ebea2 None admin] RBAC: Authorizing `identity:get_user(user_id=9f460e459de440ae86e8e754ba4941b1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:01.179040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01cbf2-9bd0-4147-8bb1-ee328e7ebea2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:01.180646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 569/1151] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:09:01 2022] GET /identity/v3/users/9f460e459de440ae86e8e754ba4941b1 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:01.186387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a5536-1f7a-4796-aeb9-f05978ca66f9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:01.285248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220a5536-1f7a-4796-aeb9-f05978ca66f9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:01.318957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c3abe-9357-4127-9f83-2b0f8f07c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:01.319579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c3abe-9357-4127-9f83-2b0f8f07c508 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:01.319892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c3abe-9357-4127-9f83-2b0f8f07c508 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:01.320170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c3abe-9357-4127-9f83-2b0f8f07c508 None admin] PATH_INFO: `/v3/projects/ae3a7bdb06c94e1688734f0f8f1f65be` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:01.321284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c3abe-9357-4127-9f83-2b0f8f07c508 None admin] RBAC: Authorizing `identity:get_project(project_id=ae3a7bdb06c94e1688734f0f8f1f65be)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:01.322713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c3abe-9357-4127-9f83-2b0f8f07c508 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:01.324369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 583/1152] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:09:01 2022] GET /identity/v3/projects/ae3a7bdb06c94e1688734f0f8f1f65be => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:01.330307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01cbf2-9bd0-4147-8bb1-ee328e7ebea2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:01.428290 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01cbf2-9bd0-4147-8bb1-ee328e7ebea2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:01.459969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37e50f-4f43-47ba-8418-81db64d0169b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:01.460619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37e50f-4f43-47ba-8418-81db64d0169b None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:01.460900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37e50f-4f43-47ba-8418-81db64d0169b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:01.461175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37e50f-4f43-47ba-8418-81db64d0169b None admin] PATH_INFO: `/v3/projects/ae3a7bdb06c94e1688734f0f8f1f65be/users/9f460e459de440ae86e8e754ba4941b1/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:01.463073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37e50f-4f43-47ba-8418-81db64d0169b None admin] RBAC: Authorizing `identity:create_grant(project_id=ae3a7bdb06c94e1688734f0f8f1f65be, user_id=9f460e459de440ae86e8e754ba4941b1, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:01.464536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e37e50f-4f43-47ba-8418-81db64d0169b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:01.478401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 570/1153] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:09:01 2022] PUT /identity/v3/projects/ae3a7bdb06c94e1688734f0f8f1f65be/users/9f460e459de440ae86e8e754ba4941b1/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:09:03.600091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d8b94-2ac6-4c8e-b2c0-d794f6155a60 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:03.600652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d8b94-2ac6-4c8e-b2c0-d794f6155a60 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:03.600652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d8b94-2ac6-4c8e-b2c0-d794f6155a60 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:03.601957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 584/1154] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:09:03.607111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90240de-7a0c-47d8-9db3-eb793c6b2178 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:03.607501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90240de-7a0c-47d8-9db3-eb793c6b2178 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:03.607661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90240de-7a0c-47d8-9db3-eb793c6b2178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:03.625869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e90240de-7a0c-47d8-9db3-eb793c6b2178 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:09:03.627433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e90240de-7a0c-47d8-9db3-eb793c6b2178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:09:03.650048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:03.699345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 571/1155] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:09:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:09:03.971975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d8b94-2ac6-4c8e-b2c0-d794f6155a60 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:03.987136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:04.072857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d8b94-2ac6-4c8e-b2c0-d794f6155a60 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:04.105685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55945d-a1b5-4148-a178-042c8de9699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:04.106304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55945d-a1b5-4148-a178-042c8de9699f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:04.106585 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55945d-a1b5-4148-a178-042c8de9699f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:04.106860 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55945d-a1b5-4148-a178-042c8de9699f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:04.117644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55945d-a1b5-4148-a178-042c8de9699f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-dc55945d-a1b5-4148-a178-042c8de9699f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:09:04.122460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:09:04.123654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:09:04.124682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 585/1156] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:09:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:09:04.129941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90240de-7a0c-47d8-9db3-eb793c6b2178 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:04.227625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90240de-7a0c-47d8-9db3-eb793c6b2178 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:04.262388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42077c6-c835-4d9c-be3d-a1d6c00d6cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:04.263338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42077c6-c835-4d9c-be3d-a1d6c00d6cfe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:04.263626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42077c6-c835-4d9c-be3d-a1d6c00d6cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:04.263929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42077c6-c835-4d9c-be3d-a1d6c00d6cfe None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:04.264607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42077c6-c835-4d9c-be3d-a1d6c00d6cfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:09:04.264912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42077c6-c835-4d9c-be3d-a1d6c00d6cfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:04.266284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42077c6-c835-4d9c-be3d-a1d6c00d6cfe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:04.276212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 572/1157] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:09:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:04.284765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55945d-a1b5-4148-a178-042c8de9699f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:04.356372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:04.387554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55945d-a1b5-4148-a178-042c8de9699f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:04.420357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f4fe5-138e-4903-a32f-488645758da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:04.420971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f4fe5-138e-4903-a32f-488645758da0 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:04.421248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f4fe5-138e-4903-a32f-488645758da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:04.421517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f4fe5-138e-4903-a32f-488645758da0 None admin] PATH_INFO: `/v3/users/9f460e459de440ae86e8e754ba4941b1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:04.422618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f4fe5-138e-4903-a32f-488645758da0 None admin] RBAC: Authorizing `identity:get_user(user_id=9f460e459de440ae86e8e754ba4941b1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:04.424057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f4fe5-138e-4903-a32f-488645758da0 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:04.425622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 586/1158] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:09:04 2022] GET /identity/v3/users/9f460e459de440ae86e8e754ba4941b1 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:04.432231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42077c6-c835-4d9c-be3d-a1d6c00d6cfe None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:04.451102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:04.530866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42077c6-c835-4d9c-be3d-a1d6c00d6cfe None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:04.563314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983716a-8064-44dc-b190-4534d39919c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:04.563942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983716a-8064-44dc-b190-4534d39919c1 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:04.564222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983716a-8064-44dc-b190-4534d39919c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:04.564495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983716a-8064-44dc-b190-4534d39919c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:04.632121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983716a-8064-44dc-b190-4534d39919c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:04.633557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983716a-8064-44dc-b190-4534d39919c1 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:04.652577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:04.730487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 573/1159] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:09:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:09:04.736599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f4fe5-138e-4903-a32f-488645758da0 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:04.834532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f4fe5-138e-4903-a32f-488645758da0 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:04.867111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41093fc5-8cd4-4a8e-bff0-d3ca2a71f507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:04.867723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41093fc5-8cd4-4a8e-bff0-d3ca2a71f507 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:04.868035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41093fc5-8cd4-4a8e-bff0-d3ca2a71f507 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:04.868309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41093fc5-8cd4-4a8e-bff0-d3ca2a71f507 None admin] PATH_INFO: `/v3/projects/ae3a7bdb06c94e1688734f0f8f1f65be` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:04.869417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41093fc5-8cd4-4a8e-bff0-d3ca2a71f507 None admin] RBAC: Authorizing `identity:get_project(project_id=ae3a7bdb06c94e1688734f0f8f1f65be)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:04.870842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41093fc5-8cd4-4a8e-bff0-d3ca2a71f507 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:04.872490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 587/1160] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:09:04 2022] GET /identity/v3/projects/ae3a7bdb06c94e1688734f0f8f1f65be => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:04.878747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983716a-8064-44dc-b190-4534d39919c1 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:04.973904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983716a-8064-44dc-b190-4534d39919c1 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:04.990223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:05.006900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fe812-f900-42b6-acb6-0c8bb9a15244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:05.007493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fe812-f900-42b6-acb6-0c8bb9a15244 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:05.007782 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fe812-f900-42b6-acb6-0c8bb9a15244 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:05.008063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fe812-f900-42b6-acb6-0c8bb9a15244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:05.008775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fe812-f900-42b6-acb6-0c8bb9a15244 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=ae3a7bdb06c94e1688734f0f8f1f65be, user.id=9f460e459de440ae86e8e754ba4941b1) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:09:05.009079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fe812-f900-42b6-acb6-0c8bb9a15244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:05.010488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fe812-f900-42b6-acb6-0c8bb9a15244 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:05.016045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 574/1161] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:09:04 2022] GET /identity/v3/role_assignments?user.id=9f460e459de440ae86e8e754ba4941b1&scope.project.id=ae3a7bdb06c94e1688734f0f8f1f65be&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:07.166410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6bda3-f879-4997-aaa4-517639f9323b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:07.166981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6bda3-f879-4997-aaa4-517639f9323b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:07.166981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6bda3-f879-4997-aaa4-517639f9323b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:07.168305 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 588/1162] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:09:07.173582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ee2a9-8993-4d62-8133-08fc306a1ff3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:07.173962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ee2a9-8993-4d62-8133-08fc306a1ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:07.174134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ee2a9-8993-4d62-8133-08fc306a1ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:07.192512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fd4ee2a9-8993-4d62-8133-08fc306a1ff3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:09:07.193696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fd4ee2a9-8993-4d62-8133-08fc306a1ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:09:07.258136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 575/1163] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:09:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:09:07.534289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6bda3-f879-4997-aaa4-517639f9323b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:07.546228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:07.638636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6bda3-f879-4997-aaa4-517639f9323b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:07.672201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1057279-746c-4fda-a312-2ccb3111da89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:07.672796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057279-746c-4fda-a312-2ccb3111da89 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:07.673076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057279-746c-4fda-a312-2ccb3111da89 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:07.673348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1057279-746c-4fda-a312-2ccb3111da89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:07.737998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1057279-746c-4fda-a312-2ccb3111da89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:07.739435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1057279-746c-4fda-a312-2ccb3111da89 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:07.749259 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:07.839176 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 589/1164] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:09:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:09:07.845377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ee2a9-8993-4d62-8133-08fc306a1ff3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:07.941651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ee2a9-8993-4d62-8133-08fc306a1ff3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:07.954707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:07.976263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64633b0-419a-4d93-8475-7f89cb641e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:07.976887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64633b0-419a-4d93-8475-7f89cb641e6f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:07.977167 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64633b0-419a-4d93-8475-7f89cb641e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:07.977435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64633b0-419a-4d93-8475-7f89cb641e6f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:07.983824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64633b0-419a-4d93-8475-7f89cb641e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:07.985245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64633b0-419a-4d93-8475-7f89cb641e6f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c64633b0-419a-4d93-8475-7f89cb641e6f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:09:07.996302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:09:07.997475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:09:07.998359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 576/1165] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:09:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:09:08.003507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1057279-746c-4fda-a312-2ccb3111da89 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:08.101244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1057279-746c-4fda-a312-2ccb3111da89 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:08.112279 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:08.137926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7133616-a38b-4eab-b395-285595199121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:08.138540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7133616-a38b-4eab-b395-285595199121 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:08.138813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7133616-a38b-4eab-b395-285595199121 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:08.139085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7133616-a38b-4eab-b395-285595199121 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:08.139718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7133616-a38b-4eab-b395-285595199121 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:09:08.140048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7133616-a38b-4eab-b395-285595199121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:08.141408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7133616-a38b-4eab-b395-285595199121 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:08.148434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 590/1166] 10.222.0.71 () {56 vars in 1115 bytes} [Wed Sep 21 12:09:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:10.253073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d951a-977c-4f4a-94c5-c8f5a9e06772 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:10.253610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d951a-977c-4f4a-94c5-c8f5a9e06772 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:10.253610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d951a-977c-4f4a-94c5-c8f5a9e06772 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:10.254924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 577/1167] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:09:10.260122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52aa4-c163-42af-a182-df1684ba2e48 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:10.260504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52aa4-c163-42af-a182-df1684ba2e48 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:10.260674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52aa4-c163-42af-a182-df1684ba2e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:10.278349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-87b52aa4-c163-42af-a182-df1684ba2e48 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:09:10.279523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-87b52aa4-c163-42af-a182-df1684ba2e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:09:10.345503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 591/1168] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:09:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:09:10.622036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d951a-977c-4f4a-94c5-c8f5a9e06772 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:10.720909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7d951a-977c-4f4a-94c5-c8f5a9e06772 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:10.753185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c80f56-b07e-4208-83de-0c69693505ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:10.753791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c80f56-b07e-4208-83de-0c69693505ea None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:10.754071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c80f56-b07e-4208-83de-0c69693505ea None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:10.754339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c80f56-b07e-4208-83de-0c69693505ea None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:10.755016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c80f56-b07e-4208-83de-0c69693505ea None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:10.756490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c80f56-b07e-4208-83de-0c69693505ea None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:10.772696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 578/1169] 10.222.0.71 () {60 vars in 1138 bytes} [Wed Sep 21 12:09:10 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:09:12.918260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221acb6a-6a6c-4e83-acbd-717e8fb13e44 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:12.918699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221acb6a-6a6c-4e83-acbd-717e8fb13e44 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:12.918813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221acb6a-6a6c-4e83-acbd-717e8fb13e44 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:12.920229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 592/1170] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:09:12.925584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092739e9-5f1b-4d84-96f8-d3c5ec0b69a2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:12.925999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092739e9-5f1b-4d84-96f8-d3c5ec0b69a2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:12.926145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092739e9-5f1b-4d84-96f8-d3c5ec0b69a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:12.946791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-092739e9-5f1b-4d84-96f8-d3c5ec0b69a2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:09:12.948050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-092739e9-5f1b-4d84-96f8-d3c5ec0b69a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:09:12.974680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:13.017448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 579/1171] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:09:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:09:13.290810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221acb6a-6a6c-4e83-acbd-717e8fb13e44 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:13.390381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221acb6a-6a6c-4e83-acbd-717e8fb13e44 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:13.391941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:13.425053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbbf61b-d05e-4e0c-9969-4ca69645ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:13.425673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbf61b-d05e-4e0c-9969-4ca69645ae14 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:13.425955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbf61b-d05e-4e0c-9969-4ca69645ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:13.426227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbf61b-d05e-4e0c-9969-4ca69645ae14 None admin] PATH_INFO: `/v3/domains/fe12442e2f1d4291b378b618941cc8ff` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:13.436959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbf61b-d05e-4e0c-9969-4ca69645ae14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe12442e2f1d4291b378b618941cc8ff)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:13.438404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbf61b-d05e-4e0c-9969-4ca69645ae14 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:13.440019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 593/1172] 10.222.0.71 () {56 vars in 1183 bytes} [Wed Sep 21 12:09:13 2022] GET /identity/v3/domains/fe12442e2f1d4291b378b618941cc8ff => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:13.445903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092739e9-5f1b-4d84-96f8-d3c5ec0b69a2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:13.541512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092739e9-5f1b-4d84-96f8-d3c5ec0b69a2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:13.574560 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a65793-a614-4fa0-b3e3-ad5907ca2f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:13.575165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a65793-a614-4fa0-b3e3-ad5907ca2f5b None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:13.575441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a65793-a614-4fa0-b3e3-ad5907ca2f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:13.575718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a65793-a614-4fa0-b3e3-ad5907ca2f5b None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:13.576401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a65793-a614-4fa0-b3e3-ad5907ca2f5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:13.577814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a65793-a614-4fa0-b3e3-ad5907ca2f5b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:13.603924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 580/1173] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:09:13 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:09:15.782445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc44382-87d4-4256-8372-290c10f14221 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:15.782927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc44382-87d4-4256-8372-290c10f14221 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:15.783020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc44382-87d4-4256-8372-290c10f14221 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:15.784353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 594/1174] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:09:15.790336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77691cef-f00b-4614-bb73-799874ba0aef None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:15.790764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77691cef-f00b-4614-bb73-799874ba0aef None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:15.790893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77691cef-f00b-4614-bb73-799874ba0aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:15.808753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-77691cef-f00b-4614-bb73-799874ba0aef None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:09:15.809939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-77691cef-f00b-4614-bb73-799874ba0aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:09:15.818933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:15.879164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 581/1175] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:09:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:09:16.150893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc44382-87d4-4256-8372-290c10f14221 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:16.252717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc44382-87d4-4256-8372-290c10f14221 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:16.285700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e35ca-b911-4037-9d1f-f59f93dd6e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:16.286342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e35ca-b911-4037-9d1f-f59f93dd6e82 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:16.286626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e35ca-b911-4037-9d1f-f59f93dd6e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:16.286909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e35ca-b911-4037-9d1f-f59f93dd6e82 None admin] PATH_INFO: `/v3/domains/fe12442e2f1d4291b378b618941cc8ff` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:16.288051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e35ca-b911-4037-9d1f-f59f93dd6e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe12442e2f1d4291b378b618941cc8ff)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:16.289494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3e35ca-b911-4037-9d1f-f59f93dd6e82 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:16.291071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 595/1176] 10.222.0.71 () {56 vars in 1183 bytes} [Wed Sep 21 12:09:16 2022] GET /identity/v3/domains/fe12442e2f1d4291b378b618941cc8ff => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:16.296844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77691cef-f00b-4614-bb73-799874ba0aef None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:16.392554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77691cef-f00b-4614-bb73-799874ba0aef None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:16.425620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53baf6cd-fac3-4ac5-95ad-6e35a17b8e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:16.426160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53baf6cd-fac3-4ac5-95ad-6e35a17b8e7f None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:16.426440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53baf6cd-fac3-4ac5-95ad-6e35a17b8e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:16.426710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53baf6cd-fac3-4ac5-95ad-6e35a17b8e7f None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:16.427399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53baf6cd-fac3-4ac5-95ad-6e35a17b8e7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:16.428840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53baf6cd-fac3-4ac5-95ad-6e35a17b8e7f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:16.445425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:16.453276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 582/1177] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:09:16 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:09:18.657265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f41483-90c5-429f-87f8-cc915752bec7 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:18.657827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f41483-90c5-429f-87f8-cc915752bec7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:18.657827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f41483-90c5-429f-87f8-cc915752bec7 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:18.659161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 596/1178] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:09:18.665133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea702d-1d7b-41d6-8b00-c54a5a02855a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:18.665560 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea702d-1d7b-41d6-8b00-c54a5a02855a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:18.665664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea702d-1d7b-41d6-8b00-c54a5a02855a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:18.683986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6aea702d-1d7b-41d6-8b00-c54a5a02855a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:09:18.685170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6aea702d-1d7b-41d6-8b00-c54a5a02855a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:09:18.750436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 583/1179] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:09:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:09:19.026739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f41483-90c5-429f-87f8-cc915752bec7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:19.061753 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:19.133143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f41483-90c5-429f-87f8-cc915752bec7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:19.165963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9272fcc-1b94-4835-bd9e-04a911bed22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:19.166586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9272fcc-1b94-4835-bd9e-04a911bed22d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:19.166867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9272fcc-1b94-4835-bd9e-04a911bed22d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:19.167137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9272fcc-1b94-4835-bd9e-04a911bed22d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:19.178055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9272fcc-1b94-4835-bd9e-04a911bed22d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-a9272fcc-1b94-4835-bd9e-04a911bed22d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:09:19.182758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:09:19.183930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:09:19.184880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 597/1180] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:09:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:09:19.189936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea702d-1d7b-41d6-8b00-c54a5a02855a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:19.285766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea702d-1d7b-41d6-8b00-c54a5a02855a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:19.316698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:19.322940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e6139-8fee-4568-888c-ba49bcaa11de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:19.323544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e6139-8fee-4568-888c-ba49bcaa11de None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:19.323848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e6139-8fee-4568-888c-ba49bcaa11de None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:19.324121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e6139-8fee-4568-888c-ba49bcaa11de None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:19.324778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e6139-8fee-4568-888c-ba49bcaa11de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:09:19.325077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e6139-8fee-4568-888c-ba49bcaa11de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:19.326436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e6139-8fee-4568-888c-ba49bcaa11de None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:19.334551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 584/1181] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:09:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:19.341059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9272fcc-1b94-4835-bd9e-04a911bed22d None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:19.432682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:19.442202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9272fcc-1b94-4835-bd9e-04a911bed22d None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:19.475036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fe820-93c3-468d-999e-e72ab2db5662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:19.475659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fe820-93c3-468d-999e-e72ab2db5662 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:19.475966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fe820-93c3-468d-999e-e72ab2db5662 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:19.476241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fe820-93c3-468d-999e-e72ab2db5662 None admin] PATH_INFO: `/v3/users/ca4856e264cc4217b9faf58e9510c96f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:19.488814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6fe820-93c3-468d-999e-e72ab2db5662 None admin] RBAC: Authorizing `identity:get_user(user_id=ca4856e264cc4217b9faf58e9510c96f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:19.490874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6fe820-93c3-468d-999e-e72ab2db5662 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:19.492500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 598/1182] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:09:19 2022] GET /identity/v3/users/ca4856e264cc4217b9faf58e9510c96f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:19.498364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e6139-8fee-4568-888c-ba49bcaa11de None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:19.596937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e6139-8fee-4568-888c-ba49bcaa11de None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:19.629454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36627c-c3d1-4e24-ad48-4b9ada4dba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:19.630089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36627c-c3d1-4e24-ad48-4b9ada4dba99 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:19.630487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36627c-c3d1-4e24-ad48-4b9ada4dba99 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:19.630884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36627c-c3d1-4e24-ad48-4b9ada4dba99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:19.697627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:19.710113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36627c-c3d1-4e24-ad48-4b9ada4dba99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:19.711558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36627c-c3d1-4e24-ad48-4b9ada4dba99 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:19.806601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 585/1183] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:09:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:09:19.812571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fe820-93c3-468d-999e-e72ab2db5662 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:19.909654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fe820-93c3-468d-999e-e72ab2db5662 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:19.943343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b3989-aa36-490f-87e7-87e513f4c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:19.943984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b3989-aa36-490f-87e7-87e513f4c446 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:19.944263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b3989-aa36-490f-87e7-87e513f4c446 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:19.944534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b3989-aa36-490f-87e7-87e513f4c446 None admin] PATH_INFO: `/v3/projects/7f0a8e4c676547f1a8728e7b0e414258` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:19.945636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b3989-aa36-490f-87e7-87e513f4c446 None admin] RBAC: Authorizing `identity:get_project(project_id=7f0a8e4c676547f1a8728e7b0e414258)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:19.947071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b3989-aa36-490f-87e7-87e513f4c446 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:19.948737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 599/1184] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:09:19 2022] GET /identity/v3/projects/7f0a8e4c676547f1a8728e7b0e414258 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:19.954903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36627c-c3d1-4e24-ad48-4b9ada4dba99 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:20.047325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:20.056904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36627c-c3d1-4e24-ad48-4b9ada4dba99 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:20.091434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b41f68-22fb-44af-bbc4-8fcda6647290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:20.092053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b41f68-22fb-44af-bbc4-8fcda6647290 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:20.092332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b41f68-22fb-44af-bbc4-8fcda6647290 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:20.092602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b41f68-22fb-44af-bbc4-8fcda6647290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:20.093320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b41f68-22fb-44af-bbc4-8fcda6647290 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=7f0a8e4c676547f1a8728e7b0e414258, user.id=ca4856e264cc4217b9faf58e9510c96f) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:09:20.093626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b41f68-22fb-44af-bbc4-8fcda6647290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:20.095034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b41f68-22fb-44af-bbc4-8fcda6647290 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:20.100781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 586/1185] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:09:19 2022] GET /identity/v3/role_assignments?user.id=ca4856e264cc4217b9faf58e9510c96f&scope.project.id=7f0a8e4c676547f1a8728e7b0e414258&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:22.269006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c447a-f71b-422f-9383-a9ba44e78518 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:22.269578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c447a-f71b-422f-9383-a9ba44e78518 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:22.269578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c447a-f71b-422f-9383-a9ba44e78518 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:22.271477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 600/1186] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:09:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:09:22.276626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7204510-8957-406f-96ce-2f39cb202e4d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:22.277031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7204510-8957-406f-96ce-2f39cb202e4d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:22.277183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7204510-8957-406f-96ce-2f39cb202e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:22.294861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d7204510-8957-406f-96ce-2f39cb202e4d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:09:22.296090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d7204510-8957-406f-96ce-2f39cb202e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:09:22.366560 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 587/1187] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:09:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:09:22.644491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c447a-f71b-422f-9383-a9ba44e78518 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:22.662714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:22.755733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c447a-f71b-422f-9383-a9ba44e78518 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:22.790085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb7589-6bfd-4adb-8d7d-83dceeeddcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:22.790699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb7589-6bfd-4adb-8d7d-83dceeeddcad None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:22.790975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb7589-6bfd-4adb-8d7d-83dceeeddcad None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:22.791244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb7589-6bfd-4adb-8d7d-83dceeeddcad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:22.801052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb7589-6bfd-4adb-8d7d-83dceeeddcad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-10eb7589-6bfd-4adb-8d7d-83dceeeddcad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:09:22.805783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:09:22.806906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:09:22.807970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 601/1188] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:09:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:09:22.813055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7204510-8957-406f-96ce-2f39cb202e4d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:22.893696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:22.908969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7204510-8957-406f-96ce-2f39cb202e4d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:22.942134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de636574-f023-40c6-85ea-30c988ad1d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:22.943239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de636574-f023-40c6-85ea-30c988ad1d93 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:22.943514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de636574-f023-40c6-85ea-30c988ad1d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:22.943804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de636574-f023-40c6-85ea-30c988ad1d93 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:22.944462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de636574-f023-40c6-85ea-30c988ad1d93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:09:22.944766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de636574-f023-40c6-85ea-30c988ad1d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:22.946126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de636574-f023-40c6-85ea-30c988ad1d93 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:22.953896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 588/1189] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:09:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:22.960597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb7589-6bfd-4adb-8d7d-83dceeeddcad None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:23.041106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:23.064135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb7589-6bfd-4adb-8d7d-83dceeeddcad None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:23.098424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996dc1d-a8fa-4fa4-894f-329d66b66ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:23.099064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996dc1d-a8fa-4fa4-894f-329d66b66ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:23.099346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996dc1d-a8fa-4fa4-894f-329d66b66ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:23.099622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996dc1d-a8fa-4fa4-894f-329d66b66ac4 None admin] PATH_INFO: `/v3/users/ca4856e264cc4217b9faf58e9510c96f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:23.100766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996dc1d-a8fa-4fa4-894f-329d66b66ac4 None admin] RBAC: Authorizing `identity:get_user(user_id=ca4856e264cc4217b9faf58e9510c96f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:23.102197 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996dc1d-a8fa-4fa4-894f-329d66b66ac4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:23.103795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 602/1190] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:09:22 2022] GET /identity/v3/users/ca4856e264cc4217b9faf58e9510c96f => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:23.109897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de636574-f023-40c6-85ea-30c988ad1d93 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:23.206730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de636574-f023-40c6-85ea-30c988ad1d93 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:23.239607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c6268-17a5-4fc7-a4fc-86754bf34898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:23.240262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c6268-17a5-4fc7-a4fc-86754bf34898 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:23.240540 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c6268-17a5-4fc7-a4fc-86754bf34898 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:23.240810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c6268-17a5-4fc7-a4fc-86754bf34898 None admin] PATH_INFO: `/v3/projects/7f0a8e4c676547f1a8728e7b0e414258` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:23.241908 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c6268-17a5-4fc7-a4fc-86754bf34898 None admin] RBAC: Authorizing `identity:get_project(project_id=7f0a8e4c676547f1a8728e7b0e414258)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:23.243336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c6268-17a5-4fc7-a4fc-86754bf34898 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:23.244988 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 589/1191] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:09:23 2022] GET /identity/v3/projects/7f0a8e4c676547f1a8728e7b0e414258 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:23.250972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996dc1d-a8fa-4fa4-894f-329d66b66ac4 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:23.350397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996dc1d-a8fa-4fa4-894f-329d66b66ac4 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:23.384488 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0016201-1ef7-44d1-9037-70ddfa1b8c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:23.385130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0016201-1ef7-44d1-9037-70ddfa1b8c6f None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:23.385412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0016201-1ef7-44d1-9037-70ddfa1b8c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:23.385684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0016201-1ef7-44d1-9037-70ddfa1b8c6f None admin] PATH_INFO: `/v3/projects/7f0a8e4c676547f1a8728e7b0e414258/users/ca4856e264cc4217b9faf58e9510c96f/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:23.387593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0016201-1ef7-44d1-9037-70ddfa1b8c6f None admin] RBAC: Authorizing `identity:create_grant(project_id=7f0a8e4c676547f1a8728e7b0e414258, user_id=ca4856e264cc4217b9faf58e9510c96f, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:23.389051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0016201-1ef7-44d1-9037-70ddfa1b8c6f None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:23.401718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 603/1192] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:09:23 2022] PUT /identity/v3/projects/7f0a8e4c676547f1a8728e7b0e414258/users/ca4856e264cc4217b9faf58e9510c96f/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:09:25.497520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a92e8-47a7-4edb-b95b-752647f749e5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:25.498114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a92e8-47a7-4edb-b95b-752647f749e5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:25.498114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a92e8-47a7-4edb-b95b-752647f749e5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:25.499394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 590/1193] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:09:25.505053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e2098-af87-4881-8e6a-79517e5bf0df None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:25.505483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e2098-af87-4881-8e6a-79517e5bf0df None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:25.505620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e2098-af87-4881-8e6a-79517e5bf0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:25.512932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:25.525448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-167e2098-af87-4881-8e6a-79517e5bf0df None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:09:25.526626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-167e2098-af87-4881-8e6a-79517e5bf0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:09:25.590573 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 604/1194] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:09:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:09:25.854732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a92e8-47a7-4edb-b95b-752647f749e5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:25.874340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:25.958371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a92e8-47a7-4edb-b95b-752647f749e5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:25.990477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c15538-fe3e-497e-9d0c-8d5224472a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:25.991094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c15538-fe3e-497e-9d0c-8d5224472a5d None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:25.991374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c15538-fe3e-497e-9d0c-8d5224472a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:25.991649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c15538-fe3e-497e-9d0c-8d5224472a5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:26.001917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c15538-fe3e-497e-9d0c-8d5224472a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-43c15538-fe3e-497e-9d0c-8d5224472a5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:09:26.008669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:09:26.009815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:09:26.010989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 591/1195] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:09:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:09:26.016359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e2098-af87-4881-8e6a-79517e5bf0df None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:26.116221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e2098-af87-4881-8e6a-79517e5bf0df None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:26.149188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcdf7e-d96a-4706-8b39-f52eeeaa286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:26.149783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcdf7e-d96a-4706-8b39-f52eeeaa286c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:26.150060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcdf7e-d96a-4706-8b39-f52eeeaa286c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:26.150328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcdf7e-d96a-4706-8b39-f52eeeaa286c None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:26.150986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcdf7e-d96a-4706-8b39-f52eeeaa286c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:09:26.151284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcdf7e-d96a-4706-8b39-f52eeeaa286c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:26.152680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcdf7e-d96a-4706-8b39-f52eeeaa286c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:26.153974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:26.161836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 605/1196] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:26.167812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c15538-fe3e-497e-9d0c-8d5224472a5d None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:26.245758 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:26.268339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c15538-fe3e-497e-9d0c-8d5224472a5d None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:26.300789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada8942-daeb-4d1c-b1f8-def6a3bb8124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:26.301411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8942-daeb-4d1c-b1f8-def6a3bb8124 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:26.302283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8942-daeb-4d1c-b1f8-def6a3bb8124 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:26.302283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada8942-daeb-4d1c-b1f8-def6a3bb8124 None admin] PATH_INFO: `/v3/users/ca4856e264cc4217b9faf58e9510c96f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:26.303049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada8942-daeb-4d1c-b1f8-def6a3bb8124 None admin] RBAC: Authorizing `identity:get_user(user_id=ca4856e264cc4217b9faf58e9510c96f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:26.304497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada8942-daeb-4d1c-b1f8-def6a3bb8124 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:26.306042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 592/1197] 10.222.0.71 () {56 vars in 1177 bytes} [Wed Sep 21 12:09:26 2022] GET /identity/v3/users/ca4856e264cc4217b9faf58e9510c96f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:26.311837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcdf7e-d96a-4706-8b39-f52eeeaa286c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:26.417239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcdf7e-d96a-4706-8b39-f52eeeaa286c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:26.449952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3a951-8c96-4d3b-80d6-35c27df7bef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:26.450545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3a951-8c96-4d3b-80d6-35c27df7bef6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:26.450819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3a951-8c96-4d3b-80d6-35c27df7bef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:26.451087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3a951-8c96-4d3b-80d6-35c27df7bef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:26.509384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:26.516908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3a951-8c96-4d3b-80d6-35c27df7bef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:26.518425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3a951-8c96-4d3b-80d6-35c27df7bef6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:26.615499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 606/1198] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:09:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:09:26.621628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada8942-daeb-4d1c-b1f8-def6a3bb8124 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:26.727536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada8942-daeb-4d1c-b1f8-def6a3bb8124 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:26.761080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e4d81-bc31-4803-a819-8694c71c3864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:26.762222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e4d81-bc31-4803-a819-8694c71c3864 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:26.762222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e4d81-bc31-4803-a819-8694c71c3864 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:26.762415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e4d81-bc31-4803-a819-8694c71c3864 None admin] PATH_INFO: `/v3/projects/7f0a8e4c676547f1a8728e7b0e414258` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:26.763379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e4d81-bc31-4803-a819-8694c71c3864 None admin] RBAC: Authorizing `identity:get_project(project_id=7f0a8e4c676547f1a8728e7b0e414258)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:26.764842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e4d81-bc31-4803-a819-8694c71c3864 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:26.766483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 593/1199] 10.222.0.71 () {56 vars in 1186 bytes} [Wed Sep 21 12:09:26 2022] GET /identity/v3/projects/7f0a8e4c676547f1a8728e7b0e414258 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:09:26.773395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3a951-8c96-4d3b-80d6-35c27df7bef6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:09:26.864256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:26.870642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3a951-8c96-4d3b-80d6-35c27df7bef6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:09:26.906123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fd851-6f12-4105-bc16-5721edd5305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:09:26.906731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd851-6f12-4105-bc16-5721edd5305c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:09:26.907014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd851-6f12-4105-bc16-5721edd5305c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:09:26.907304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fd851-6f12-4105-bc16-5721edd5305c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:09:26.908038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fd851-6f12-4105-bc16-5721edd5305c None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=7f0a8e4c676547f1a8728e7b0e414258, user.id=ca4856e264cc4217b9faf58e9510c96f) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:09:26.908343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fd851-6f12-4105-bc16-5721edd5305c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:09:26.909768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0fd851-6f12-4105-bc16-5721edd5305c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:09:26.915540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 607/1200] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:09:26 2022] GET /identity/v3/role_assignments?user.id=ca4856e264cc4217b9faf58e9510c96f&scope.project.id=7f0a8e4c676547f1a8728e7b0e414258&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:09:36.757771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:09:36.912954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=20 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:23.361675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f3c1e-c473-4dad-af63-0c964679fa46 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:23.361675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f3c1e-c473-4dad-af63-0c964679fa46 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:23.361675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f3c1e-c473-4dad-af63-0c964679fa46 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:23.364332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 594/1201] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:10:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:10:23.371689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb75c90-ebc4-4ccf-9a1b-ef0c423abb1f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:23.372308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb75c90-ebc4-4ccf-9a1b-ef0c423abb1f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:23.372308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb75c90-ebc4-4ccf-9a1b-ef0c423abb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:23.397560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1eb75c90-ebc4-4ccf-9a1b-ef0c423abb1f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:10:23.398757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1eb75c90-ebc4-4ccf-9a1b-ef0c423abb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:10:23.491824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 608/1202] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:10:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:10:23.805594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f3c1e-c473-4dad-af63-0c964679fa46 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:23.941166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f3c1e-c473-4dad-af63-0c964679fa46 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:23.982661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92b037-238f-4f0e-aa26-c05e4bad5c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:23.983614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92b037-238f-4f0e-aa26-c05e4bad5c08 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:23.984066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92b037-238f-4f0e-aa26-c05e4bad5c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:23.984474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92b037-238f-4f0e-aa26-c05e4bad5c08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:24.008336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92b037-238f-4f0e-aa26-c05e4bad5c08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:24.009821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92b037-238f-4f0e-aa26-c05e4bad5c08 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-4f92b037-238f-4f0e-aa26-c05e4bad5c08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:24.027043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:24.028409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:24.029745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:24.029745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:24.029745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:24.034247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 595/1203] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:24.035310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb75c90-ebc4-4ccf-9a1b-ef0c423abb1f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:24.170254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb75c90-ebc4-4ccf-9a1b-ef0c423abb1f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:24.220262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:24.228281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d2635-80e8-4df7-a393-775f33aeaa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:24.228880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d2635-80e8-4df7-a393-775f33aeaa5e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:24.229160 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d2635-80e8-4df7-a393-775f33aeaa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:24.229434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d2635-80e8-4df7-a393-775f33aeaa5e None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:24.230088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d2635-80e8-4df7-a393-775f33aeaa5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:24.230391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d2635-80e8-4df7-a393-775f33aeaa5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:24.231791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d2635-80e8-4df7-a393-775f33aeaa5e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:24.245681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 609/1204] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:24.251666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92b037-238f-4f0e-aa26-c05e4bad5c08 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:24.317137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:24.373732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92b037-238f-4f0e-aa26-c05e4bad5c08 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:24.424272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ca855-3d63-460d-b469-cba22254eea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:24.425056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca855-3d63-460d-b469-cba22254eea8 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:24.425254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca855-3d63-460d-b469-cba22254eea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:24.425563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca855-3d63-460d-b469-cba22254eea8 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:24.426258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ca855-3d63-460d-b469-cba22254eea8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:24.428053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ca855-3d63-460d-b469-cba22254eea8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:24.458355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 596/1205] 10.222.0.71 () {60 vars in 1132 bytes} [Wed Sep 21 12:10:24 2022] POST /identity/v3/users => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:10:26.833510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d318e0e-70c2-4059-98b5-3156b274efb4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:26.834111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d318e0e-70c2-4059-98b5-3156b274efb4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:26.834111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d318e0e-70c2-4059-98b5-3156b274efb4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:26.835529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 610/1206] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:10:26.842034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d70c4a-bf96-4a79-b1b2-01d1a938b2a5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:26.842525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d70c4a-bf96-4a79-b1b2-01d1a938b2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:26.842612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d70c4a-bf96-4a79-b1b2-01d1a938b2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:26.863326 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b6d70c4a-bf96-4a79-b1b2-01d1a938b2a5 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:10:26.864614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b6d70c4a-bf96-4a79-b1b2-01d1a938b2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:10:26.960100 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 597/1207] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:10:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:10:27.257642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d318e0e-70c2-4059-98b5-3156b274efb4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:27.382859 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d318e0e-70c2-4059-98b5-3156b274efb4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:27.434254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6057fe-c9f7-420f-9d40-1d5aa532ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:27.434920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6057fe-c9f7-420f-9d40-1d5aa532ff0f None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:27.435225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6057fe-c9f7-420f-9d40-1d5aa532ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:27.435474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6057fe-c9f7-420f-9d40-1d5aa532ff0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:27.449761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6057fe-c9f7-420f-9d40-1d5aa532ff0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-4c6057fe-c9f7-420f-9d40-1d5aa532ff0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:10:27.455171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:27.456283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:27.460922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 611/1208] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:10:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:10:27.463004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d70c4a-bf96-4a79-b1b2-01d1a938b2a5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:27.505495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:27.575129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d70c4a-bf96-4a79-b1b2-01d1a938b2a5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:27.617627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc342d-24ae-4045-acc0-50484a0645a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:27.618407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc342d-24ae-4045-acc0-50484a0645a2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:27.618781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc342d-24ae-4045-acc0-50484a0645a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:27.619050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc342d-24ae-4045-acc0-50484a0645a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:27.619863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc342d-24ae-4045-acc0-50484a0645a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:27.620264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc342d-24ae-4045-acc0-50484a0645a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:27.622127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc342d-24ae-4045-acc0-50484a0645a2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:27.633832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 598/1209] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:10:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:10:27.640096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6057fe-c9f7-420f-9d40-1d5aa532ff0f None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:27.702706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:27.773783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6057fe-c9f7-420f-9d40-1d5aa532ff0f None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:27.814587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7bfd5-c536-4bfb-ae97-49377fd9b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:27.815275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7bfd5-c536-4bfb-ae97-49377fd9b310 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:27.815628 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7bfd5-c536-4bfb-ae97-49377fd9b310 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:27.815960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7bfd5-c536-4bfb-ae97-49377fd9b310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:27.829093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7bfd5-c536-4bfb-ae97-49377fd9b310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:27.830703 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7bfd5-c536-4bfb-ae97-49377fd9b310 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-5db7bfd5-c536-4bfb-ae97-49377fd9b310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:27.847564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:27.854067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:27.855074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:27.855074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:27.855074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:27.855074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 612/1210] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:27.857115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc342d-24ae-4045-acc0-50484a0645a2 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:27.975656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:27.978779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc342d-24ae-4045-acc0-50484a0645a2 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:28.021838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32ec97-01b5-402b-9987-71de6b8f4ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:28.022456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32ec97-01b5-402b-9987-71de6b8f4ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:28.022734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32ec97-01b5-402b-9987-71de6b8f4ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:28.023001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32ec97-01b5-402b-9987-71de6b8f4ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:28.023637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32ec97-01b5-402b-9987-71de6b8f4ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:28.023994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32ec97-01b5-402b-9987-71de6b8f4ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:28.025353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32ec97-01b5-402b-9987-71de6b8f4ac7 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:28.038221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 599/1211] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:28.043828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7bfd5-c536-4bfb-ae97-49377fd9b310 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:28.150522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7bfd5-c536-4bfb-ae97-49377fd9b310 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:28.152034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:28.186232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8c542-9f13-45a8-a9d8-82f515b20bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:28.186959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8c542-9f13-45a8-a9d8-82f515b20bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:28.187243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8c542-9f13-45a8-a9d8-82f515b20bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:28.187523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c8c542-9f13-45a8-a9d8-82f515b20bb6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:28.196482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8c542-9f13-45a8-a9d8-82f515b20bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:28.198083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c8c542-9f13-45a8-a9d8-82f515b20bb6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-38c8c542-9f13-45a8-a9d8-82f515b20bb6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:10:28.211409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:28.212734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:28.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 613/1212] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:28.219721 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32ec97-01b5-402b-9987-71de6b8f4ac7 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:28.340821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32ec97-01b5-402b-9987-71de6b8f4ac7 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:28.381055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1865d7-379f-4f03-99b7-f5356d59e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:28.382190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1865d7-379f-4f03-99b7-f5356d59e121 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:28.382771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1865d7-379f-4f03-99b7-f5356d59e121 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:28.383356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1865d7-379f-4f03-99b7-f5356d59e121 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:28.384519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1865d7-379f-4f03-99b7-f5356d59e121 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:28.385171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1865d7-379f-4f03-99b7-f5356d59e121 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:28.387410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1865d7-379f-4f03-99b7-f5356d59e121 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:28.416791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 600/1213] 10.222.0.71 () {56 vars in 1143 bytes} [Wed Sep 21 12:10:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:28.425680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8c542-9f13-45a8-a9d8-82f515b20bb6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:28.567384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c8c542-9f13-45a8-a9d8-82f515b20bb6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:28.614266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff8071-7009-4d96-9092-b0170cc6fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:28.614765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff8071-7009-4d96-9092-b0170cc6fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:28.615108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff8071-7009-4d96-9092-b0170cc6fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:28.615392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff8071-7009-4d96-9092-b0170cc6fbc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:28.661267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:28.712534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff8071-7009-4d96-9092-b0170cc6fbc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:28.714409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff8071-7009-4d96-9092-b0170cc6fbc9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:28.851461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 614/1214] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:10:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:10:28.857429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1865d7-379f-4f03-99b7-f5356d59e121 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:28.877733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:28.976844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1865d7-379f-4f03-99b7-f5356d59e121 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:29.014707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df167fdb-a7f0-4830-bb37-9b7420c888ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:29.015360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df167fdb-a7f0-4830-bb37-9b7420c888ac None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:29.015640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df167fdb-a7f0-4830-bb37-9b7420c888ac None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:29.016139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df167fdb-a7f0-4830-bb37-9b7420c888ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:29.028983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df167fdb-a7f0-4830-bb37-9b7420c888ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:29.030482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df167fdb-a7f0-4830-bb37-9b7420c888ac None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-df167fdb-a7f0-4830-bb37-9b7420c888ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:29.049407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:29.050543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:29.051428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:29.051428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:29.051428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:29.051428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 601/1215] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:29.056991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff8071-7009-4d96-9092-b0170cc6fbc9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:29.146455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:29.192574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff8071-7009-4d96-9092-b0170cc6fbc9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:29.231990 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb052bf-6d9d-45e5-99ac-9eb5e9a35ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:29.232633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb052bf-6d9d-45e5-99ac-9eb5e9a35ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:29.232899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb052bf-6d9d-45e5-99ac-9eb5e9a35ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:29.233319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb052bf-6d9d-45e5-99ac-9eb5e9a35ef2 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:29.234163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb052bf-6d9d-45e5-99ac-9eb5e9a35ef2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:29.234480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb052bf-6d9d-45e5-99ac-9eb5e9a35ef2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:29.236944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb052bf-6d9d-45e5-99ac-9eb5e9a35ef2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:29.251452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 615/1216] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:29.257681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df167fdb-a7f0-4830-bb37-9b7420c888ac None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:29.335788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:29.371464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df167fdb-a7f0-4830-bb37-9b7420c888ac None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:29.424506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2275dc1-44b7-45f8-ac2b-81d485a298dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:29.425679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2275dc1-44b7-45f8-ac2b-81d485a298dc None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:29.426139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2275dc1-44b7-45f8-ac2b-81d485a298dc None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:29.426675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2275dc1-44b7-45f8-ac2b-81d485a298dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:29.438907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2275dc1-44b7-45f8-ac2b-81d485a298dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:29.441744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2275dc1-44b7-45f8-ac2b-81d485a298dc None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-e2275dc1-44b7-45f8-ac2b-81d485a298dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:10:29.456382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:29.457624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:29.458762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:10:29.458762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:10:29.458762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:10:29.458762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:10:29.458762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:10:29.458762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:10:29.458762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:29.458762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:29.458762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 602/1217] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:10:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:29.464074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb052bf-6d9d-45e5-99ac-9eb5e9a35ef2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:29.582154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb052bf-6d9d-45e5-99ac-9eb5e9a35ef2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:29.608677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:29.624024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5116fd2-1069-4fea-abc5-5b412204dee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:29.624710 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5116fd2-1069-4fea-abc5-5b412204dee4 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:29.625019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5116fd2-1069-4fea-abc5-5b412204dee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:29.625290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5116fd2-1069-4fea-abc5-5b412204dee4 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:29.626055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5116fd2-1069-4fea-abc5-5b412204dee4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:29.626540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5116fd2-1069-4fea-abc5-5b412204dee4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:29.628185 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5116fd2-1069-4fea-abc5-5b412204dee4 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:29.642321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 616/1218] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:10:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:29.649136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2275dc1-44b7-45f8-ac2b-81d485a298dc None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:29.765900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2275dc1-44b7-45f8-ac2b-81d485a298dc None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:29.799195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:29.806060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c18e80-a09b-44b0-94be-777d488f894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:29.806665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c18e80-a09b-44b0-94be-777d488f894f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:29.806935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c18e80-a09b-44b0-94be-777d488f894f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:29.807229 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c18e80-a09b-44b0-94be-777d488f894f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:29.808222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c18e80-a09b-44b0-94be-777d488f894f None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=eac29fd43f1f4bbd987950b90385200b) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:29.808527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c18e80-a09b-44b0-94be-777d488f894f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:29.810022 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c18e80-a09b-44b0-94be-777d488f894f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:29.823470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 603/1219] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:10:29 2022] GET /identity/v3/role_assignments?user.id=eac29fd43f1f4bbd987950b90385200b&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:10:32.044072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a304f5-260e-4346-b589-5828f53ac888 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:32.044629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a304f5-260e-4346-b589-5828f53ac888 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:32.044629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a304f5-260e-4346-b589-5828f53ac888 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:32.045904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 617/1220] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:10:32.051568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873d08f-69e7-4121-9f76-41877e529c0e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:32.052048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873d08f-69e7-4121-9f76-41877e529c0e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:32.052219 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873d08f-69e7-4121-9f76-41877e529c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:32.071573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5873d08f-69e7-4121-9f76-41877e529c0e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:10:32.072805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5873d08f-69e7-4121-9f76-41877e529c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:10:32.148533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 604/1221] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:10:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:10:32.426684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a304f5-260e-4346-b589-5828f53ac888 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:32.541200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a304f5-260e-4346-b589-5828f53ac888 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:32.579067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfdda2-1fbb-40d7-9593-ebd30bf686ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:32.579719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfdda2-1fbb-40d7-9593-ebd30bf686ff None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:32.580032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfdda2-1fbb-40d7-9593-ebd30bf686ff None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:32.580309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfdda2-1fbb-40d7-9593-ebd30bf686ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:32.591130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfdda2-1fbb-40d7-9593-ebd30bf686ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-1bdfdda2-1fbb-40d7-9593-ebd30bf686ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:10:32.596108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:32.597245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:32.598276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 618/1222] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:10:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:32.603521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873d08f-69e7-4121-9f76-41877e529c0e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:32.711076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873d08f-69e7-4121-9f76-41877e529c0e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:32.742209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:32.754625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8c9f3-28cb-46cf-b095-6d83be0542a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:32.755250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8c9f3-28cb-46cf-b095-6d83be0542a4 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:32.755532 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8c9f3-28cb-46cf-b095-6d83be0542a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:32.755862 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8c9f3-28cb-46cf-b095-6d83be0542a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:32.756553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8c9f3-28cb-46cf-b095-6d83be0542a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:32.756891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8c9f3-28cb-46cf-b095-6d83be0542a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:32.759728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8c9f3-28cb-46cf-b095-6d83be0542a4 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:32.771789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 605/1223] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:10:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:32.773906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfdda2-1fbb-40d7-9593-ebd30bf686ff None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:32.823162 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:32.882987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfdda2-1fbb-40d7-9593-ebd30bf686ff None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:32.918021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c0f3b-7931-42e9-9a22-b3ccf29b8ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:32.918653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c0f3b-7931-42e9-9a22-b3ccf29b8ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:32.918939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c0f3b-7931-42e9-9a22-b3ccf29b8ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:32.919214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c0f3b-7931-42e9-9a22-b3ccf29b8ac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:32.929982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c0f3b-7931-42e9-9a22-b3ccf29b8ac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:32.931427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c0f3b-7931-42e9-9a22-b3ccf29b8ac9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-692c0f3b-7931-42e9-9a22-b3ccf29b8ac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:32.946235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:32.947219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:32.948496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:32.948496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:32.948496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:32.948496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 619/1224] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:32.953290 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8c9f3-28cb-46cf-b095-6d83be0542a4 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:33.064589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8c9f3-28cb-46cf-b095-6d83be0542a4 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:33.102025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b77b0-c191-43b8-892b-633975233d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:33.102837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b77b0-c191-43b8-892b-633975233d58 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:33.103190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b77b0-c191-43b8-892b-633975233d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:33.103503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b77b0-c191-43b8-892b-633975233d58 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:33.104335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b77b0-c191-43b8-892b-633975233d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:33.104778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b77b0-c191-43b8-892b-633975233d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:33.106459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b77b0-c191-43b8-892b-633975233d58 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:33.122182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 606/1225] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:33.131612 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c0f3b-7931-42e9-9a22-b3ccf29b8ac9 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:33.244964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:33.253260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c0f3b-7931-42e9-9a22-b3ccf29b8ac9 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:33.290671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f07cb9-6657-451c-9e98-58addf0a6fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:33.291346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f07cb9-6657-451c-9e98-58addf0a6fad None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:33.291641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f07cb9-6657-451c-9e98-58addf0a6fad None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:33.291942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f07cb9-6657-451c-9e98-58addf0a6fad None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:33.301544 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f07cb9-6657-451c-9e98-58addf0a6fad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:33.303074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f07cb9-6657-451c-9e98-58addf0a6fad None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-e8f07cb9-6657-451c-9e98-58addf0a6fad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:10:33.317570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:33.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:33.320073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 620/1226] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:33.325165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b77b0-c191-43b8-892b-633975233d58 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:33.378273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:33.447511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b77b0-c191-43b8-892b-633975233d58 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:33.484425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30ddc8-3c34-474a-94aa-c2c226b756a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:33.485034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30ddc8-3c34-474a-94aa-c2c226b756a6 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:33.485333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30ddc8-3c34-474a-94aa-c2c226b756a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:33.485630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30ddc8-3c34-474a-94aa-c2c226b756a6 None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:33.486371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30ddc8-3c34-474a-94aa-c2c226b756a6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:33.486742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30ddc8-3c34-474a-94aa-c2c226b756a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:33.489003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30ddc8-3c34-474a-94aa-c2c226b756a6 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:33.511828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 607/1227] 10.222.0.71 () {56 vars in 1143 bytes} [Wed Sep 21 12:10:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:33.513163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f07cb9-6657-451c-9e98-58addf0a6fad None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:33.622561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f07cb9-6657-451c-9e98-58addf0a6fad None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:33.664124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bbbb-58c1-4e1e-8b04-dee7d1c4fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:33.664778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bbbb-58c1-4e1e-8b04-dee7d1c4fa34 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:33.665090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bbbb-58c1-4e1e-8b04-dee7d1c4fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:33.665407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bbbb-58c1-4e1e-8b04-dee7d1c4fa34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:33.677265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32bbbb-58c1-4e1e-8b04-dee7d1c4fa34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:33.679292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32bbbb-58c1-4e1e-8b04-dee7d1c4fa34 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-ff32bbbb-58c1-4e1e-8b04-dee7d1c4fa34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:33.695097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:33.696637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:33.697732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:33.697732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:33.697732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:33.697732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 621/1228] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:33.702265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30ddc8-3c34-474a-94aa-c2c226b756a6 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:33.818805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30ddc8-3c34-474a-94aa-c2c226b756a6 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:33.824157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:33.862544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acfa92-c115-4779-889b-d47bcdd17eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:33.863178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acfa92-c115-4779-889b-d47bcdd17eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:33.863457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acfa92-c115-4779-889b-d47bcdd17eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:33.863885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acfa92-c115-4779-889b-d47bcdd17eb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:33.864566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acfa92-c115-4779-889b-d47bcdd17eb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:33.865005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acfa92-c115-4779-889b-d47bcdd17eb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:33.866417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acfa92-c115-4779-889b-d47bcdd17eb8 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:33.882663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 608/1229] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:33.889077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bbbb-58c1-4e1e-8b04-dee7d1c4fa34 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:33.898976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:33.998476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bbbb-58c1-4e1e-8b04-dee7d1c4fa34 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:34.036667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61623912-c4b8-4e0e-a3ee-0c4fc8f95ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:34.037308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61623912-c4b8-4e0e-a3ee-0c4fc8f95ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:34.037583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61623912-c4b8-4e0e-a3ee-0c4fc8f95ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:34.037858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61623912-c4b8-4e0e-a3ee-0c4fc8f95ad6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:34.044899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61623912-c4b8-4e0e-a3ee-0c4fc8f95ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:34.046426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61623912-c4b8-4e0e-a3ee-0c4fc8f95ad6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-61623912-c4b8-4e0e-a3ee-0c4fc8f95ad6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:10:34.058576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:34.059643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:34.060582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:10:34.060582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:10:34.060582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:10:34.060582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:10:34.060582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:10:34.060582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:10:34.060582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:34.060582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:34.060582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 622/1230] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:10:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:34.066603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acfa92-c115-4779-889b-d47bcdd17eb8 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:34.172240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acfa92-c115-4779-889b-d47bcdd17eb8 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:34.210425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d4e3c-39c6-4383-8802-5ad2b935d626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:34.211051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d4e3c-39c6-4383-8802-5ad2b935d626 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:34.211330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d4e3c-39c6-4383-8802-5ad2b935d626 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:34.211600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d4e3c-39c6-4383-8802-5ad2b935d626 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:34.212329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d4e3c-39c6-4383-8802-5ad2b935d626 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:34.212650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d4e3c-39c6-4383-8802-5ad2b935d626 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:34.214058 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d4e3c-39c6-4383-8802-5ad2b935d626 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:34.227074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 609/1231] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:10:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:34.233161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61623912-c4b8-4e0e-a3ee-0c4fc8f95ad6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:34.309210 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:34.355849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61623912-c4b8-4e0e-a3ee-0c4fc8f95ad6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:34.396460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0849b-3652-453a-835c-5a4901d82cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:34.397189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0849b-3652-453a-835c-5a4901d82cc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:34.397520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0849b-3652-453a-835c-5a4901d82cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:34.397791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0849b-3652-453a-835c-5a4901d82cc2 None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/eac29fd43f1f4bbd987950b90385200b/roles/fcc43e29f89e4c6cb1690a36d91ec4c1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:34.415749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0849b-3652-453a-835c-5a4901d82cc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=eac29fd43f1f4bbd987950b90385200b, role_id=fcc43e29f89e4c6cb1690a36d91ec4c1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:34.417391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0849b-3652-453a-835c-5a4901d82cc2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:34.430854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 623/1232] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:10:34 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/eac29fd43f1f4bbd987950b90385200b/roles/fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:10:36.637611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd8874-f177-49db-bd7b-0616c667e76c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:36.638206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd8874-f177-49db-bd7b-0616c667e76c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:36.638206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd8874-f177-49db-bd7b-0616c667e76c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:36.639483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 610/1233] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:10:36.645651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa476-6476-4d82-bc1e-1b35f698d1cb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:36.646187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa476-6476-4d82-bc1e-1b35f698d1cb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:36.646377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa476-6476-4d82-bc1e-1b35f698d1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:36.667576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cdeaa476-6476-4d82-bc1e-1b35f698d1cb None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:10:36.668921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cdeaa476-6476-4d82-bc1e-1b35f698d1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:10:36.742950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 624/1234] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:10:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:10:37.053157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd8874-f177-49db-bd7b-0616c667e76c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:37.081372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:37.173704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd8874-f177-49db-bd7b-0616c667e76c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:37.218586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4a155-428c-4279-b5d7-2b9fcd967c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:37.219398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4a155-428c-4279-b5d7-2b9fcd967c01 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:37.220303 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4a155-428c-4279-b5d7-2b9fcd967c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:37.220580 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4a155-428c-4279-b5d7-2b9fcd967c01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:37.233660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4a155-428c-4279-b5d7-2b9fcd967c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-69f4a155-428c-4279-b5d7-2b9fcd967c01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:10:37.239794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:37.241113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:37.242021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 611/1235] 10.222.0.71 () {56 vars in 1102 bytes} [Wed Sep 21 12:10:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:37.247645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaa476-6476-4d82-bc1e-1b35f698d1cb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:37.299930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:37.368092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaa476-6476-4d82-bc1e-1b35f698d1cb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:37.416926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e57437-a2cb-484a-a296-f71c4362db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:37.417734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e57437-a2cb-484a-a296-f71c4362db6e None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:37.418152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e57437-a2cb-484a-a296-f71c4362db6e None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:37.418567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e57437-a2cb-484a-a296-f71c4362db6e None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:37.419403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e57437-a2cb-484a-a296-f71c4362db6e None admin] RBAC: Adding query filter params (name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:37.420100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e57437-a2cb-484a-a296-f71c4362db6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:37.422143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e57437-a2cb-484a-a296-f71c4362db6e None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:37.439632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 625/1236] 10.222.0.71 () {56 vars in 1103 bytes} [Wed Sep 21 12:10:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:37.440798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4a155-428c-4279-b5d7-2b9fcd967c01 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:37.527610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:37.544028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4a155-428c-4279-b5d7-2b9fcd967c01 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:37.581440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeda687-0ae0-4aff-8baa-489fe82a239a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:37.582073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeda687-0ae0-4aff-8baa-489fe82a239a None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:37.582351 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeda687-0ae0-4aff-8baa-489fe82a239a None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:37.582620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeda687-0ae0-4aff-8baa-489fe82a239a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:37.593598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeda687-0ae0-4aff-8baa-489fe82a239a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:37.595162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeda687-0ae0-4aff-8baa-489fe82a239a None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-fdeda687-0ae0-4aff-8baa-489fe82a239a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:37.613814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:37.614931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:37.615869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:37.615869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:37.615869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:37.615869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 612/1237] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:37.621439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e57437-a2cb-484a-a296-f71c4362db6e None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:37.731683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e57437-a2cb-484a-a296-f71c4362db6e None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:37.746168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:37.773759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89517cf6-65e8-4642-a69d-5d6352680530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:37.774555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89517cf6-65e8-4642-a69d-5d6352680530 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:37.774865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89517cf6-65e8-4642-a69d-5d6352680530 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:37.775258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89517cf6-65e8-4642-a69d-5d6352680530 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:37.775982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89517cf6-65e8-4642-a69d-5d6352680530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:37.776315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89517cf6-65e8-4642-a69d-5d6352680530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:37.777889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89517cf6-65e8-4642-a69d-5d6352680530 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:37.791997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 626/1238] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:10:37.797943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeda687-0ae0-4aff-8baa-489fe82a239a None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:37.918928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeda687-0ae0-4aff-8baa-489fe82a239a None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:37.957192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d9b4-70c6-4e05-8fb9-0fc832d92ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:37.957889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d9b4-70c6-4e05-8fb9-0fc832d92ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:37.958208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d9b4-70c6-4e05-8fb9-0fc832d92ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:37.958561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4d9b4-70c6-4e05-8fb9-0fc832d92ed9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:37.961714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:37.967523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4d9b4-70c6-4e05-8fb9-0fc832d92ed9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:37.969358 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df4d9b4-70c6-4e05-8fb9-0fc832d92ed9 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-2df4d9b4-70c6-4e05-8fb9-0fc832d92ed9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:10:37.980579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:37.981742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:37.983109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 613/1239] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:10:37.988133 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89517cf6-65e8-4642-a69d-5d6352680530 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:38.093650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89517cf6-65e8-4642-a69d-5d6352680530 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:38.133208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299d072-8bf6-455d-8f70-c46f1b29a1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:38.133829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299d072-8bf6-455d-8f70-c46f1b29a1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:38.134105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299d072-8bf6-455d-8f70-c46f1b29a1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:38.134382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299d072-8bf6-455d-8f70-c46f1b29a1b7 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:38.135130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299d072-8bf6-455d-8f70-c46f1b29a1b7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:38.135427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299d072-8bf6-455d-8f70-c46f1b29a1b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:38.137237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299d072-8bf6-455d-8f70-c46f1b29a1b7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:38.157394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 627/1240] 10.222.0.71 () {56 vars in 1143 bytes} [Wed Sep 21 12:10:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:38.163780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d9b4-70c6-4e05-8fb9-0fc832d92ed9 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:38.281858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df4d9b4-70c6-4e05-8fb9-0fc832d92ed9 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:38.319214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec3737-190d-4225-a3d4-1420172c3d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:38.319858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec3737-190d-4225-a3d4-1420172c3d47 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:38.320142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec3737-190d-4225-a3d4-1420172c3d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:38.320413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec3737-190d-4225-a3d4-1420172c3d47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:38.392726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:38.400168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec3737-190d-4225-a3d4-1420172c3d47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:38.401633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec3737-190d-4225-a3d4-1420172c3d47 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:38.506640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 614/1241] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:10:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 12:10:38.512567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299d072-8bf6-455d-8f70-c46f1b29a1b7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:38.548952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:38.640556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299d072-8bf6-455d-8f70-c46f1b29a1b7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:38.681546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e4178-453e-4ed5-8047-fd383c35e2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:38.682179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e4178-453e-4ed5-8047-fd383c35e2ea None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:38.682446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e4178-453e-4ed5-8047-fd383c35e2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:38.682716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e4178-453e-4ed5-8047-fd383c35e2ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:38.693336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e4178-453e-4ed5-8047-fd383c35e2ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:38.694780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e4178-453e-4ed5-8047-fd383c35e2ea None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2d8e4178-453e-4ed5-8047-fd383c35e2ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:38.709835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:38.710994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:38.711922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:38.711922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:38.711922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:38.711922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 628/1242] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:38.717596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec3737-190d-4225-a3d4-1420172c3d47 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:38.830047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:38.837864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec3737-190d-4225-a3d4-1420172c3d47 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:38.874352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e8b69-70f3-4c83-94b2-8457a5fddffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:38.875192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e8b69-70f3-4c83-94b2-8457a5fddffe None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:38.875470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e8b69-70f3-4c83-94b2-8457a5fddffe None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:38.876330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e8b69-70f3-4c83-94b2-8457a5fddffe None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:38.877038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e8b69-70f3-4c83-94b2-8457a5fddffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:38.877486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e8b69-70f3-4c83-94b2-8457a5fddffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:38.879682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e8b69-70f3-4c83-94b2-8457a5fddffe None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:38.893910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 615/1243] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:38.900235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e4178-453e-4ed5-8047-fd383c35e2ea None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:38.998330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:39.025303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e4178-453e-4ed5-8047-fd383c35e2ea None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:39.060455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b2d21-c1b1-412b-9f6d-18cde8a8cad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:39.061230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b2d21-c1b1-412b-9f6d-18cde8a8cad2 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:39.061625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b2d21-c1b1-412b-9f6d-18cde8a8cad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:39.062016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b2d21-c1b1-412b-9f6d-18cde8a8cad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:39.069828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b2d21-c1b1-412b-9f6d-18cde8a8cad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:39.071462 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b2d21-c1b1-412b-9f6d-18cde8a8cad2 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-8a0b2d21-c1b1-412b-9f6d-18cde8a8cad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:10:39.085465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:39.086719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:39.092662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:10:39.092662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:10:39.092662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:10:39.092662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:10:39.092662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:10:39.092662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:10:39.092662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:39.092662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:39.092662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 629/1244] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:10:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:39.093512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e8b69-70f3-4c83-94b2-8457a5fddffe None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:39.203805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e8b69-70f3-4c83-94b2-8457a5fddffe None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:39.243409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f29fb5-13e5-4f8d-b1ca-5549819b57b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:39.245828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29fb5-13e5-4f8d-b1ca-5549819b57b2 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:39.246192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29fb5-13e5-4f8d-b1ca-5549819b57b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:39.246487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f29fb5-13e5-4f8d-b1ca-5549819b57b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:39.247278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29fb5-13e5-4f8d-b1ca-5549819b57b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:39.247582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29fb5-13e5-4f8d-b1ca-5549819b57b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:39.249025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f29fb5-13e5-4f8d-b1ca-5549819b57b2 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:39.262397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 616/1245] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:10:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:39.268958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b2d21-c1b1-412b-9f6d-18cde8a8cad2 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:39.379456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b2d21-c1b1-412b-9f6d-18cde8a8cad2 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:39.424447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f517f-ff69-4a9a-ad0f-8d0041d3c8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:39.425226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f517f-ff69-4a9a-ad0f-8d0041d3c8ba None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:39.425499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f517f-ff69-4a9a-ad0f-8d0041d3c8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:39.425866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f517f-ff69-4a9a-ad0f-8d0041d3c8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:39.426890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f517f-ff69-4a9a-ad0f-8d0041d3c8ba None admin] RBAC: Adding query filter params (role.id=fcc43e29f89e4c6cb1690a36d91ec4c1, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=eac29fd43f1f4bbd987950b90385200b) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:39.427266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f517f-ff69-4a9a-ad0f-8d0041d3c8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:39.429112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f517f-ff69-4a9a-ad0f-8d0041d3c8ba None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:39.436721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 630/1246] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:10:39 2022] GET /identity/v3/role_assignments?user.id=eac29fd43f1f4bbd987950b90385200b&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=fcc43e29f89e4c6cb1690a36d91ec4c1 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:41.708770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3661b4-0e37-461a-9901-1910df7a3814 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:41.709367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3661b4-0e37-461a-9901-1910df7a3814 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:41.709367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3661b4-0e37-461a-9901-1910df7a3814 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:41.710683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 617/1247] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:10:41.716304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f50b-2a24-4fb6-a3ff-c522b50aff7c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:41.716830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f50b-2a24-4fb6-a3ff-c522b50aff7c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:41.716830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f50b-2a24-4fb6-a3ff-c522b50aff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:41.724097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:41.739659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-11f7f50b-2a24-4fb6-a3ff-c522b50aff7c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:10:41.741097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-11f7f50b-2a24-4fb6-a3ff-c522b50aff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:10:41.827880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 631/1248] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:10:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:10:42.096211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3661b4-0e37-461a-9901-1910df7a3814 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:42.114747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:42.217400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3661b4-0e37-461a-9901-1910df7a3814 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:42.252708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5d0fb-bada-4ae4-abe5-02ffebb4fdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:42.253435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5d0fb-bada-4ae4-abe5-02ffebb4fdeb None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:42.253616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5d0fb-bada-4ae4-abe5-02ffebb4fdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:42.253873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5d0fb-bada-4ae4-abe5-02ffebb4fdeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:42.265655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5d0fb-bada-4ae4-abe5-02ffebb4fdeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-4bb5d0fb-bada-4ae4-abe5-02ffebb4fdeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:10:42.271190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:42.272236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:42.273192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 618/1249] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:10:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:42.280079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7f50b-2a24-4fb6-a3ff-c522b50aff7c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:42.416071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7f50b-2a24-4fb6-a3ff-c522b50aff7c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:42.460526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:42.466552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541cf30-e6a7-40af-9b55-e5e9d7e3918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:42.467166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541cf30-e6a7-40af-9b55-e5e9d7e3918b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:42.467448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541cf30-e6a7-40af-9b55-e5e9d7e3918b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:42.467722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541cf30-e6a7-40af-9b55-e5e9d7e3918b None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:42.468409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541cf30-e6a7-40af-9b55-e5e9d7e3918b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:42.468710 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541cf30-e6a7-40af-9b55-e5e9d7e3918b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:42.470152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541cf30-e6a7-40af-9b55-e5e9d7e3918b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:42.483785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 632/1250] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:10:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:42.486294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5d0fb-bada-4ae4-abe5-02ffebb4fdeb None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:42.583177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:42.613100 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5d0fb-bada-4ae4-abe5-02ffebb4fdeb None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:42.659884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f8e5a-158e-4cf7-9171-dce9ab16e711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:42.660511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f8e5a-158e-4cf7-9171-dce9ab16e711 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:42.660791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f8e5a-158e-4cf7-9171-dce9ab16e711 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:42.661055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f8e5a-158e-4cf7-9171-dce9ab16e711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:42.672680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f8e5a-158e-4cf7-9171-dce9ab16e711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:42.674176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f8e5a-158e-4cf7-9171-dce9ab16e711 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-c57f8e5a-158e-4cf7-9171-dce9ab16e711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:42.690181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:42.691398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:42.692469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:42.692469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:42.692469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:42.692469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 619/1251] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:42.697315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541cf30-e6a7-40af-9b55-e5e9d7e3918b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:42.812769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541cf30-e6a7-40af-9b55-e5e9d7e3918b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:42.851741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873275-5641-4156-8233-7f9327e135c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:42.852442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873275-5641-4156-8233-7f9327e135c5 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:42.852719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873275-5641-4156-8233-7f9327e135c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:42.852989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873275-5641-4156-8233-7f9327e135c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:42.853629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba873275-5641-4156-8233-7f9327e135c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:42.853961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba873275-5641-4156-8233-7f9327e135c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:42.855331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba873275-5641-4156-8233-7f9327e135c5 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:42.868846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 633/1252] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:42.874813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f8e5a-158e-4cf7-9171-dce9ab16e711 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:42.983092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f8e5a-158e-4cf7-9171-dce9ab16e711 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:43.017371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:43.026625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2f715-9fb5-4bc8-93fa-2e2b6ec9673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:43.027261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2f715-9fb5-4bc8-93fa-2e2b6ec9673c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:43.027540 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2f715-9fb5-4bc8-93fa-2e2b6ec9673c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:43.027878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2f715-9fb5-4bc8-93fa-2e2b6ec9673c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:43.037048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2f715-9fb5-4bc8-93fa-2e2b6ec9673c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:43.038511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2f715-9fb5-4bc8-93fa-2e2b6ec9673c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-9af2f715-9fb5-4bc8-93fa-2e2b6ec9673c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:10:43.050893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:43.051947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:43.052938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 620/1253] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 12:10:43.058158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873275-5641-4156-8233-7f9327e135c5 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:43.120804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:43.174998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873275-5641-4156-8233-7f9327e135c5 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:43.212131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcab940-d617-497b-a4a5-521e34ab1f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:43.212748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcab940-d617-497b-a4a5-521e34ab1f03 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:43.213024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcab940-d617-497b-a4a5-521e34ab1f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:43.213290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcab940-d617-497b-a4a5-521e34ab1f03 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:43.214028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcab940-d617-497b-a4a5-521e34ab1f03 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:43.214330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcab940-d617-497b-a4a5-521e34ab1f03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:43.216031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcab940-d617-497b-a4a5-521e34ab1f03 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:43.241007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 634/1254] 10.222.0.71 () {56 vars in 1143 bytes} [Wed Sep 21 12:10:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:10:43.242172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2f715-9fb5-4bc8-93fa-2e2b6ec9673c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:43.350624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2f715-9fb5-4bc8-93fa-2e2b6ec9673c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:43.386910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8348013-b99e-44f8-b185-77a5c5e52fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:43.387536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8348013-b99e-44f8-b185-77a5c5e52fba None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:43.387878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8348013-b99e-44f8-b185-77a5c5e52fba None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:43.388131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8348013-b99e-44f8-b185-77a5c5e52fba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:43.452248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:43.465676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8348013-b99e-44f8-b185-77a5c5e52fba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:43.467130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8348013-b99e-44f8-b185-77a5c5e52fba None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:43.567804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 621/1255] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:10:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 12:10:43.573702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcab940-d617-497b-a4a5-521e34ab1f03 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:43.676828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcab940-d617-497b-a4a5-521e34ab1f03 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:43.690622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:43.712614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50b550-0112-4a9e-b2c5-a44064dc7032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:43.713252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50b550-0112-4a9e-b2c5-a44064dc7032 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:43.713540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50b550-0112-4a9e-b2c5-a44064dc7032 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:43.713813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50b550-0112-4a9e-b2c5-a44064dc7032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:43.724842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50b550-0112-4a9e-b2c5-a44064dc7032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:43.726301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50b550-0112-4a9e-b2c5-a44064dc7032 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-da50b550-0112-4a9e-b2c5-a44064dc7032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:43.741699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:43.742729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:43.743621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:43.743621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:43.743621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:43.743621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 635/1256] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:43.750789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8348013-b99e-44f8-b185-77a5c5e52fba None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:43.836769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:43.849636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8348013-b99e-44f8-b185-77a5c5e52fba None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:43.882353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcb813-a519-4b60-b49d-a4d5174c719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:43.882979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcb813-a519-4b60-b49d-a4d5174c719f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:43.883217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcb813-a519-4b60-b49d-a4d5174c719f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:43.883486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcb813-a519-4b60-b49d-a4d5174c719f None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:43.884151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcb813-a519-4b60-b49d-a4d5174c719f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:43.884449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcb813-a519-4b60-b49d-a4d5174c719f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:43.885813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcb813-a519-4b60-b49d-a4d5174c719f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:43.897576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 622/1257] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:10:43.903401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50b550-0112-4a9e-b2c5-a44064dc7032 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:44.002879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50b550-0112-4a9e-b2c5-a44064dc7032 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:44.036718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11052426-391f-4c34-a17c-0a0467ddeb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:44.037344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11052426-391f-4c34-a17c-0a0467ddeb57 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:44.037626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11052426-391f-4c34-a17c-0a0467ddeb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:44.037901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11052426-391f-4c34-a17c-0a0467ddeb57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:44.044438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11052426-391f-4c34-a17c-0a0467ddeb57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:44.045874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11052426-391f-4c34-a17c-0a0467ddeb57 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-11052426-391f-4c34-a17c-0a0467ddeb57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:10:44.057855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:44.058722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:44.059611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:10:44.059611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:10:44.059611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:10:44.059611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:10:44.059611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:10:44.059611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:10:44.059611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:44.059611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:44.059611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 636/1258] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:44.065451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcb813-a519-4b60-b49d-a4d5174c719f None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:44.161679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcb813-a519-4b60-b49d-a4d5174c719f None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:44.193978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28688601-3dd7-47a0-89c9-ff71af1fc569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:44.194571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688601-3dd7-47a0-89c9-ff71af1fc569 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:44.194848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688601-3dd7-47a0-89c9-ff71af1fc569 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:44.195117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28688601-3dd7-47a0-89c9-ff71af1fc569 None admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:44.195819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28688601-3dd7-47a0-89c9-ff71af1fc569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:44.196122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28688601-3dd7-47a0-89c9-ff71af1fc569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:44.197532 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28688601-3dd7-47a0-89c9-ff71af1fc569 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:44.209727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 623/1259] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:44.216096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11052426-391f-4c34-a17c-0a0467ddeb57 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:44.241085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:44.314502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11052426-391f-4c34-a17c-0a0467ddeb57 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:44.359223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44d40b-a819-469a-b375-17101ff34be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:44.360366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44d40b-a819-469a-b375-17101ff34be9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:44.360645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44d40b-a819-469a-b375-17101ff34be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:44.360911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44d40b-a819-469a-b375-17101ff34be9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:44.361636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44d40b-a819-469a-b375-17101ff34be9 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=eac29fd43f1f4bbd987950b90385200b) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:44.361948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44d40b-a819-469a-b375-17101ff34be9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:44.363976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44d40b-a819-469a-b375-17101ff34be9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:44.369916 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 637/1260] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:10:44 2022] GET /identity/v3/role_assignments?user.id=eac29fd43f1f4bbd987950b90385200b&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:46.493145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82511a14-8c3b-435b-9fde-3838b1cc748b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:46.493683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82511a14-8c3b-435b-9fde-3838b1cc748b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:46.493683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82511a14-8c3b-435b-9fde-3838b1cc748b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:46.495026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 624/1261] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:10:46.500783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67872af-f473-4afe-9b2b-42f203b47928 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:46.501223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67872af-f473-4afe-9b2b-42f203b47928 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:46.501312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67872af-f473-4afe-9b2b-42f203b47928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:46.519827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b67872af-f473-4afe-9b2b-42f203b47928 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:10:46.521024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b67872af-f473-4afe-9b2b-42f203b47928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:10:46.587845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 638/1262] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:10:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:10:46.871685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82511a14-8c3b-435b-9fde-3838b1cc748b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:46.883813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:46.982626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82511a14-8c3b-435b-9fde-3838b1cc748b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:47.017915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc00c0a-45a8-4f6e-aeb7-16bae89b7391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:47.018459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc00c0a-45a8-4f6e-aeb7-16bae89b7391 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:47.018730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc00c0a-45a8-4f6e-aeb7-16bae89b7391 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:47.018998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc00c0a-45a8-4f6e-aeb7-16bae89b7391 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:47.029134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc00c0a-45a8-4f6e-aeb7-16bae89b7391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3dc00c0a-45a8-4f6e-aeb7-16bae89b7391 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:10:47.034498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:47.035609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:47.036541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 625/1263] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:10:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:47.043316 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67872af-f473-4afe-9b2b-42f203b47928 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:47.065988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:47.153344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67872af-f473-4afe-9b2b-42f203b47928 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:47.186931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e17f2-e503-4b5d-be5f-78846bf48cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:47.187532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e17f2-e503-4b5d-be5f-78846bf48cdd None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:47.188355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e17f2-e503-4b5d-be5f-78846bf48cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:47.188627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e17f2-e503-4b5d-be5f-78846bf48cdd None admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:47.189282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e17f2-e503-4b5d-be5f-78846bf48cdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:47.189580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e17f2-e503-4b5d-be5f-78846bf48cdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:47.190938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e17f2-e503-4b5d-be5f-78846bf48cdd None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:47.199245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 639/1264] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:10:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:47.205628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc00c0a-45a8-4f6e-aeb7-16bae89b7391 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:47.280880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:47.317822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc00c0a-45a8-4f6e-aeb7-16bae89b7391 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:47.351989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5c33a-41d1-48ad-8b6c-646373b9a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:47.352598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5c33a-41d1-48ad-8b6c-646373b9a3ef None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:47.352872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5c33a-41d1-48ad-8b6c-646373b9a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:47.353140 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5c33a-41d1-48ad-8b6c-646373b9a3ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:47.364257 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5c33a-41d1-48ad-8b6c-646373b9a3ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:47.365699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5c33a-41d1-48ad-8b6c-646373b9a3ef None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f9c5c33a-41d1-48ad-8b6c-646373b9a3ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:47.380964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:47.382196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:47.384182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:47.384182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:47.384182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:47.384182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 626/1265] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:47.388904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e17f2-e503-4b5d-be5f-78846bf48cdd None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:47.488361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:47.508650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e17f2-e503-4b5d-be5f-78846bf48cdd None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:47.542229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8fc7e-9afc-4c43-a8d0-2dda4320c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:47.542748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8fc7e-9afc-4c43-a8d0-2dda4320c95b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:47.543025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8fc7e-9afc-4c43-a8d0-2dda4320c95b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:47.543291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8fc7e-9afc-4c43-a8d0-2dda4320c95b None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:47.543971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8fc7e-9afc-4c43-a8d0-2dda4320c95b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:47.544270 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8fc7e-9afc-4c43-a8d0-2dda4320c95b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:47.545629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8fc7e-9afc-4c43-a8d0-2dda4320c95b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:47.557881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 640/1266] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:47.565175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5c33a-41d1-48ad-8b6c-646373b9a3ef None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:47.664384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5c33a-41d1-48ad-8b6c-646373b9a3ef None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:47.684540 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:47.698532 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd8f9e-6b1f-44f2-8cde-6bf4ae237d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:47.699203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd8f9e-6b1f-44f2-8cde-6bf4ae237d69 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:47.699489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd8f9e-6b1f-44f2-8cde-6bf4ae237d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:47.699813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd8f9e-6b1f-44f2-8cde-6bf4ae237d69 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:47.710073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd8f9e-6b1f-44f2-8cde-6bf4ae237d69 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:47.711519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd8f9e-6b1f-44f2-8cde-6bf4ae237d69 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-30fd8f9e-6b1f-44f2-8cde-6bf4ae237d69 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:10:47.723419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:47.724613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:47.725604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 627/1267] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:47.730635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8fc7e-9afc-4c43-a8d0-2dda4320c95b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:47.835102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8fc7e-9afc-4c43-a8d0-2dda4320c95b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:47.870685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db796848-b37f-4095-a46d-cbed2751da48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:47.871288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db796848-b37f-4095-a46d-cbed2751da48 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:47.871571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db796848-b37f-4095-a46d-cbed2751da48 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:47.871876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db796848-b37f-4095-a46d-cbed2751da48 None admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:47.872604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db796848-b37f-4095-a46d-cbed2751da48 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:47.872915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db796848-b37f-4095-a46d-cbed2751da48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:47.874486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db796848-b37f-4095-a46d-cbed2751da48 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:47.883079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:47.892829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 641/1268] 10.222.0.71 () {56 vars in 1143 bytes} [Wed Sep 21 12:10:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:47.898775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd8f9e-6b1f-44f2-8cde-6bf4ae237d69 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:48.005978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd8f9e-6b1f-44f2-8cde-6bf4ae237d69 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:48.040282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed685623-b57e-4bed-8290-4b22b75f666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:48.040905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed685623-b57e-4bed-8290-4b22b75f666c None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:48.041184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed685623-b57e-4bed-8290-4b22b75f666c None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:48.041454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed685623-b57e-4bed-8290-4b22b75f666c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:48.051698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed685623-b57e-4bed-8290-4b22b75f666c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:48.053160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed685623-b57e-4bed-8290-4b22b75f666c None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-ed685623-b57e-4bed-8290-4b22b75f666c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:48.066982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:48.068125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:48.069030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:48.069030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:48.069030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:48.069030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 628/1269] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:48.074049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db796848-b37f-4095-a46d-cbed2751da48 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:48.172571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db796848-b37f-4095-a46d-cbed2751da48 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:48.205497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562770e-0c12-4c6a-904d-03f106932370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:48.206087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562770e-0c12-4c6a-904d-03f106932370 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:48.206360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562770e-0c12-4c6a-904d-03f106932370 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:48.206627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562770e-0c12-4c6a-904d-03f106932370 None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:48.207250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562770e-0c12-4c6a-904d-03f106932370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:48.207546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562770e-0c12-4c6a-904d-03f106932370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:48.208919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562770e-0c12-4c6a-904d-03f106932370 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:48.220581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 642/1270] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:48.226388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed685623-b57e-4bed-8290-4b22b75f666c None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:48.257220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:48.324219 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed685623-b57e-4bed-8290-4b22b75f666c None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:48.358313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f594b67-02ab-4783-aec5-0378afa083b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:48.358937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f594b67-02ab-4783-aec5-0378afa083b3 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:48.359217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f594b67-02ab-4783-aec5-0378afa083b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:48.359517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f594b67-02ab-4783-aec5-0378afa083b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:48.366131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f594b67-02ab-4783-aec5-0378afa083b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:48.367570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f594b67-02ab-4783-aec5-0378afa083b3 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-8f594b67-02ab-4783-aec5-0378afa083b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:10:48.379466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:48.380551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:48.382319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:10:48.382319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:10:48.382319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:10:48.382319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:10:48.382319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:10:48.382319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:10:48.382319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:48.382319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:48.382319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 629/1271] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:10:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:48.387963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562770e-0c12-4c6a-904d-03f106932370 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:48.448473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:48.489717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562770e-0c12-4c6a-904d-03f106932370 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:48.523273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada27ea2-4ee6-482f-aaf9-8d3a1d77330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:48.523909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada27ea2-4ee6-482f-aaf9-8d3a1d77330d None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:48.524158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada27ea2-4ee6-482f-aaf9-8d3a1d77330d None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:48.524426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada27ea2-4ee6-482f-aaf9-8d3a1d77330d None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:48.525098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada27ea2-4ee6-482f-aaf9-8d3a1d77330d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:48.525399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada27ea2-4ee6-482f-aaf9-8d3a1d77330d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:48.526812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada27ea2-4ee6-482f-aaf9-8d3a1d77330d None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:48.538641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 643/1272] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:10:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:10:48.544711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f594b67-02ab-4783-aec5-0378afa083b3 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:48.632725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:48.642581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f594b67-02ab-4783-aec5-0378afa083b3 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:48.674302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c2bcf-a3d4-4bd8-a6ad-624e8e06ca19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:48.674953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2bcf-a3d4-4bd8-a6ad-624e8e06ca19 None admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:48.675231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2bcf-a3d4-4bd8-a6ad-624e8e06ca19 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:48.675501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c2bcf-a3d4-4bd8-a6ad-624e8e06ca19 None admin] PATH_INFO: `/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/eac29fd43f1f4bbd987950b90385200b/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:48.677392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2bcf-a3d4-4bd8-a6ad-624e8e06ca19 None admin] RBAC: Authorizing `identity:create_grant(project_id=5918af5b051047a78ca6d5f0187d5156, user_id=eac29fd43f1f4bbd987950b90385200b, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:48.678812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c2bcf-a3d4-4bd8-a6ad-624e8e06ca19 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:48.691258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 630/1273] 10.222.0.71 () {58 vars in 1439 bytes} [Wed Sep 21 12:10:48 2022] PUT /identity/v3/projects/5918af5b051047a78ca6d5f0187d5156/users/eac29fd43f1f4bbd987950b90385200b/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:10:50.840603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8db04f-f1fc-4d36-9998-9193d652c7a0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:50.841128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8db04f-f1fc-4d36-9998-9193d652c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:50.841128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8db04f-f1fc-4d36-9998-9193d652c7a0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:50.842485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 644/1274] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:10:50.847986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c9c64-970c-4dad-a4fa-c60e3632379c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:50.848425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c9c64-970c-4dad-a4fa-c60e3632379c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:50.848631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c9c64-970c-4dad-a4fa-c60e3632379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:50.866464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e37c9c64-970c-4dad-a4fa-c60e3632379c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:10:50.867646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e37c9c64-970c-4dad-a4fa-c60e3632379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:10:50.932593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 631/1275] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:10:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:10:51.202191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8db04f-f1fc-4d36-9998-9193d652c7a0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:51.304019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8db04f-f1fc-4d36-9998-9193d652c7a0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:51.320908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:51.338224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304186ce-9336-481a-a134-c066b172ea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:51.338845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304186ce-9336-481a-a134-c066b172ea5a None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:51.339127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304186ce-9336-481a-a134-c066b172ea5a None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:51.339652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304186ce-9336-481a-a134-c066b172ea5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:51.350332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304186ce-9336-481a-a134-c066b172ea5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-304186ce-9336-481a-a134-c066b172ea5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 12:10:51.355145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:51.356286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:51.357347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 645/1276] 10.222.0.71 () {56 vars in 1096 bytes} [Wed Sep 21 12:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:51.362350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c9c64-970c-4dad-a4fa-c60e3632379c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:51.458242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:51.478685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c9c64-970c-4dad-a4fa-c60e3632379c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:51.512321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b40ff-05c8-4dd9-ba1e-8c300442c057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:51.512916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b40ff-05c8-4dd9-ba1e-8c300442c057 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:51.513193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b40ff-05c8-4dd9-ba1e-8c300442c057 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:51.513459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b40ff-05c8-4dd9-ba1e-8c300442c057 None admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:51.514127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b40ff-05c8-4dd9-ba1e-8c300442c057 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:51.514636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b40ff-05c8-4dd9-ba1e-8c300442c057 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:51.516914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b40ff-05c8-4dd9-ba1e-8c300442c057 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:51.525216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 632/1277] 10.222.0.71 () {56 vars in 1099 bytes} [Wed Sep 21 12:10:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:51.531086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304186ce-9336-481a-a134-c066b172ea5a None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:51.631236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304186ce-9336-481a-a134-c066b172ea5a None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:51.664691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fbf75-a844-4d6d-a39e-60103c5d3095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:51.665314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fbf75-a844-4d6d-a39e-60103c5d3095 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:51.665692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fbf75-a844-4d6d-a39e-60103c5d3095 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:51.665973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412fbf75-a844-4d6d-a39e-60103c5d3095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:51.675816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fbf75-a844-4d6d-a39e-60103c5d3095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:51.677261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412fbf75-a844-4d6d-a39e-60103c5d3095 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-412fbf75-a844-4d6d-a39e-60103c5d3095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:51.692002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:51.693209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:51.694384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:51.694384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:51.694384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:51.694384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 646/1278] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:51.698834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b40ff-05c8-4dd9-ba1e-8c300442c057 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:51.800177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b40ff-05c8-4dd9-ba1e-8c300442c057 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:51.832537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b3265-8915-41f1-8b0e-bbf5f0deef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:51.833132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b3265-8915-41f1-8b0e-bbf5f0deef80 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:51.833412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b3265-8915-41f1-8b0e-bbf5f0deef80 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:51.833686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b3265-8915-41f1-8b0e-bbf5f0deef80 None admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:51.834325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b3265-8915-41f1-8b0e-bbf5f0deef80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:51.834626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b3265-8915-41f1-8b0e-bbf5f0deef80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:51.836014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b3265-8915-41f1-8b0e-bbf5f0deef80 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:51.847562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 633/1279] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:51.853358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fbf75-a844-4d6d-a39e-60103c5d3095 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:51.885482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:51.954773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412fbf75-a844-4d6d-a39e-60103c5d3095 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:51.994582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2239ba9-041e-432f-9785-7e357846f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:51.995511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2239ba9-041e-432f-9785-7e357846f3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:51.995964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2239ba9-041e-432f-9785-7e357846f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:51.996373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2239ba9-041e-432f-9785-7e357846f3b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:52.006001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2239ba9-041e-432f-9785-7e357846f3b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:52.007440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2239ba9-041e-432f-9785-7e357846f3b7 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-c2239ba9-041e-432f-9785-7e357846f3b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 12:10:52.018550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:52.020242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 12:10:52.022154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:52.023927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 647/1280] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:52.025457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b3265-8915-41f1-8b0e-bbf5f0deef80 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:52.029220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:52.124438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b3265-8915-41f1-8b0e-bbf5f0deef80 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:52.157580 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500039bc-b1a7-47f2-ae21-52d8683aa4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:52.158178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500039bc-b1a7-47f2-ae21-52d8683aa4ad None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:52.158455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500039bc-b1a7-47f2-ae21-52d8683aa4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:52.158724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500039bc-b1a7-47f2-ae21-52d8683aa4ad None admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:52.159447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500039bc-b1a7-47f2-ae21-52d8683aa4ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:52.159749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500039bc-b1a7-47f2-ae21-52d8683aa4ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:52.161190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500039bc-b1a7-47f2-ae21-52d8683aa4ad None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:52.178889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 634/1281] 10.222.0.71 () {56 vars in 1143 bytes} [Wed Sep 21 12:10:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:52.184699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2239ba9-041e-432f-9785-7e357846f3b7 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:52.273135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:52.283897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2239ba9-041e-432f-9785-7e357846f3b7 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:52.317298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0c0c2-bca3-4d20-b6d2-7deb34fc4a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:52.317906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0c0c2-bca3-4d20-b6d2-7deb34fc4a9b None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:52.318179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0c0c2-bca3-4d20-b6d2-7deb34fc4a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:52.318454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0c0c2-bca3-4d20-b6d2-7deb34fc4a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:52.390044 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0c0c2-bca3-4d20-b6d2-7deb34fc4a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:52.391990 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0c0c2-bca3-4d20-b6d2-7deb34fc4a9b None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:52.479662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:52.490619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 648/1282] 10.222.0.71 () {58 vars in 1282 bytes} [Wed Sep 21 12:10:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 12:10:52.496624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500039bc-b1a7-47f2-ae21-52d8683aa4ad None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:52.556681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:52.596884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500039bc-b1a7-47f2-ae21-52d8683aa4ad None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:52.634180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082cc3b-fdc2-4cf1-9813-119395a8f889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:52.634801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082cc3b-fdc2-4cf1-9813-119395a8f889 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:52.635077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082cc3b-fdc2-4cf1-9813-119395a8f889 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:52.635349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082cc3b-fdc2-4cf1-9813-119395a8f889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:52.645846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082cc3b-fdc2-4cf1-9813-119395a8f889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:52.647278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082cc3b-fdc2-4cf1-9813-119395a8f889 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-3082cc3b-fdc2-4cf1-9813-119395a8f889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 12:10:52.661357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:52.662488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 12:10:52.663554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 12:10:52.663554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 12:10:52.663554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:52.663554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 635/1283] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:52.668409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0c0c2-bca3-4d20-b6d2-7deb34fc4a9b None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:52.767123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0c0c2-bca3-4d20-b6d2-7deb34fc4a9b None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:52.800651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f172a24-8a87-4198-ab68-6f7fb5cf919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:52.801247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f172a24-8a87-4198-ab68-6f7fb5cf919c None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:52.801526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f172a24-8a87-4198-ab68-6f7fb5cf919c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:52.801798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f172a24-8a87-4198-ab68-6f7fb5cf919c None admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:52.802433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f172a24-8a87-4198-ab68-6f7fb5cf919c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:52.802741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f172a24-8a87-4198-ab68-6f7fb5cf919c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:52.804123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f172a24-8a87-4198-ab68-6f7fb5cf919c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:52.815586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 649/1284] 10.222.0.71 () {56 vars in 1109 bytes} [Wed Sep 21 12:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:52.821304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082cc3b-fdc2-4cf1-9813-119395a8f889 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:52.920373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082cc3b-fdc2-4cf1-9813-119395a8f889 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:52.934030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:52.953650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a71d1-c14c-4f1a-9211-7af63e960920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:52.954274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a71d1-c14c-4f1a-9211-7af63e960920 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:52.954552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a71d1-c14c-4f1a-9211-7af63e960920 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:52.954820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a71d1-c14c-4f1a-9211-7af63e960920 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:52.961244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a71d1-c14c-4f1a-9211-7af63e960920 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:52.962675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a71d1-c14c-4f1a-9211-7af63e960920 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application [None req-f12a71d1-c14c-4f1a-9211-7af63e960920 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 12:10:52.974331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:52.975323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:52.976627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 12:10:52.976627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 12:10:52.976627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 12:10:52.976627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 12:10:52.976627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 12:10:52.976627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 12:10:52.976627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 12:10:52.976627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: ERROR keystone.server.flask.application Sep 21 12:10:52.976627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 636/1285] 10.222.0.71 () {56 vars in 1111 bytes} [Wed Sep 21 12:10:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:52.981310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f172a24-8a87-4198-ab68-6f7fb5cf919c None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:53.032743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:53.082482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f172a24-8a87-4198-ab68-6f7fb5cf919c None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:53.114987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df294b3-4767-4efd-9c91-626a7651fe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:53.115581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df294b3-4767-4efd-9c91-626a7651fe02 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:53.115959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df294b3-4767-4efd-9c91-626a7651fe02 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:53.116237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df294b3-4767-4efd-9c91-626a7651fe02 None admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:53.116925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df294b3-4767-4efd-9c91-626a7651fe02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:53.117230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df294b3-4767-4efd-9c91-626a7651fe02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:53.118647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df294b3-4767-4efd-9c91-626a7651fe02 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:53.132904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 650/1286] 10.222.0.71 () {56 vars in 1148 bytes} [Wed Sep 21 12:10:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:53.139189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a71d1-c14c-4f1a-9211-7af63e960920 None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:53.236340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a71d1-c14c-4f1a-9211-7af63e960920 None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:53.269188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4760d-9841-4b81-ad57-dda6a3bfb777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:53.269780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4760d-9841-4b81-ad57-dda6a3bfb777 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:53.270060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4760d-9841-4b81-ad57-dda6a3bfb777 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:53.270330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4760d-9841-4b81-ad57-dda6a3bfb777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:53.271063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4760d-9841-4b81-ad57-dda6a3bfb777 None admin] RBAC: Adding query filter params (role.id=6315e2d444ce4068bdec072c623e3ec6, scope.project.id=5918af5b051047a78ca6d5f0187d5156, user.id=eac29fd43f1f4bbd987950b90385200b) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:53.271342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4760d-9841-4b81-ad57-dda6a3bfb777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:53.272791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f4760d-9841-4b81-ad57-dda6a3bfb777 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:53.278403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 637/1287] 10.222.0.71 () {56 vars in 1374 bytes} [Wed Sep 21 12:10:53 2022] GET /identity/v3/role_assignments?user.id=eac29fd43f1f4bbd987950b90385200b&scope.project.id=5918af5b051047a78ca6d5f0187d5156&role.id=6315e2d444ce4068bdec072c623e3ec6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:55.465796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d34dea-651d-43b3-a5a8-30c8f54723d2 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:55.466404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d34dea-651d-43b3-a5a8-30c8f54723d2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:55.466404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d34dea-651d-43b3-a5a8-30c8f54723d2 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:55.467688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 651/1288] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:10:55.473789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33cd11-8253-4b48-af44-723951224ee2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:55.474365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33cd11-8253-4b48-af44-723951224ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:55.474365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33cd11-8253-4b48-af44-723951224ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:55.494171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4d33cd11-8253-4b48-af44-723951224ee2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:10:55.495398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4d33cd11-8253-4b48-af44-723951224ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:10:55.525110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:55.561010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 638/1289] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:10:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 12:10:55.823698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d34dea-651d-43b3-a5a8-30c8f54723d2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:55.925055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d34dea-651d-43b3-a5a8-30c8f54723d2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:55.935571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:55.959941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec6d53-4b72-4f89-9cd8-2edb907cfdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:55.960565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec6d53-4b72-4f89-9cd8-2edb907cfdca None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:55.960844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec6d53-4b72-4f89-9cd8-2edb907cfdca None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:55.961117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec6d53-4b72-4f89-9cd8-2edb907cfdca None admin] PATH_INFO: `/v3/services/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:55.961705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec6d53-4b72-4f89-9cd8-2edb907cfdca None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:55.963062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec6d53-4b72-4f89-9cd8-2edb907cfdca None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-baec6d53-4b72-4f89-9cd8-2edb907cfdca None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:10:55.972191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:10:55.973365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:10:55.974658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 652/1290] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:10:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:10:55.979338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33cd11-8253-4b48-af44-723951224ee2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:56.075445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33cd11-8253-4b48-af44-723951224ee2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:56.109568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae664-4092-4c09-be66-6b3064595654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:56.110166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae664-4092-4c09-be66-6b3064595654 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:56.110446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae664-4092-4c09-be66-6b3064595654 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:56.110715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae664-4092-4c09-be66-6b3064595654 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:56.111340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae664-4092-4c09-be66-6b3064595654 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:56.111639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae664-4092-4c09-be66-6b3064595654 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:56.113026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae664-4092-4c09-be66-6b3064595654 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:56.117762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 639/1291] 10.222.0.71 () {56 vars in 1116 bytes} [Wed Sep 21 12:10:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:56.123783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec6d53-4b72-4f89-9cd8-2edb907cfdca None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:56.222691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec6d53-4b72-4f89-9cd8-2edb907cfdca None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:56.255297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7c211-1d82-4862-b80a-ddf06ef9c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:56.255920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c211-1d82-4862-b80a-ddf06ef9c780 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:56.256200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c211-1d82-4862-b80a-ddf06ef9c780 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:56.256470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7c211-1d82-4862-b80a-ddf06ef9c780 None admin] PATH_INFO: `/v3/services` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:56.257082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c211-1d82-4862-b80a-ddf06ef9c780 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:10:56.257381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c211-1d82-4862-b80a-ddf06ef9c780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:56.258731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7c211-1d82-4862-b80a-ddf06ef9c780 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:56.262985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 653/1292] 10.222.0.71 () {56 vars in 1116 bytes} [Wed Sep 21 12:10:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:10:58.400511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb72679-bedf-4e84-a636-c7461d58edd6 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:58.401102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb72679-bedf-4e84-a636-c7461d58edd6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:58.401102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb72679-bedf-4e84-a636-c7461d58edd6 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:58.402393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 640/1293] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:10:58.407655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d995d-2a15-4700-97ca-b83807262029 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:58.408123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d995d-2a15-4700-97ca-b83807262029 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:58.408248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d995d-2a15-4700-97ca-b83807262029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:58.426394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-812d995d-2a15-4700-97ca-b83807262029 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:10:58.427588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-812d995d-2a15-4700-97ca-b83807262029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:10:58.457668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:58.495218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 654/1294] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:10:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:10:58.767356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb72679-bedf-4e84-a636-c7461d58edd6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:10:58.797948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:10:58.868679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb72679-bedf-4e84-a636-c7461d58edd6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:10:58.902247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d2205-da43-4fbc-8e09-03e1caaccf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:10:58.902843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d2205-da43-4fbc-8e09-03e1caaccf48 None admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:10:58.903122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d2205-da43-4fbc-8e09-03e1caaccf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:10:58.903391 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d2205-da43-4fbc-8e09-03e1caaccf48 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:10:58.904051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d2205-da43-4fbc-8e09-03e1caaccf48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:10:58.905413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d2205-da43-4fbc-8e09-03e1caaccf48 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:10:58.920041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 641/1295] 10.222.0.71 () {60 vars in 1141 bytes} [Wed Sep 21 12:10:58 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:11:01.104739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e2cd4-501b-488f-9995-8c7476d59e2c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:01.105288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e2cd4-501b-488f-9995-8c7476d59e2c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:01.105288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e2cd4-501b-488f-9995-8c7476d59e2c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:01.106627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 655/1296] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:11:01.112347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8157cd5-60c1-437f-ad9f-4a75eda5fc94 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:01.112764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8157cd5-60c1-437f-ad9f-4a75eda5fc94 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:01.112901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8157cd5-60c1-437f-ad9f-4a75eda5fc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:01.131401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a8157cd5-60c1-437f-ad9f-4a75eda5fc94 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:11:01.132627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a8157cd5-60c1-437f-ad9f-4a75eda5fc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:11:01.203203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 642/1297] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:11:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:11:01.484480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e2cd4-501b-488f-9995-8c7476d59e2c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:11:01.583883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e2cd4-501b-488f-9995-8c7476d59e2c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:11:01.618340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a932ded-ae2e-4fd3-a003-1435998f3db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:11:01.618963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a932ded-ae2e-4fd3-a003-1435998f3db6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:01.619239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a932ded-ae2e-4fd3-a003-1435998f3db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:01.619506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a932ded-ae2e-4fd3-a003-1435998f3db6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:01.620104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a932ded-ae2e-4fd3-a003-1435998f3db6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:11:01.621457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a932ded-ae2e-4fd3-a003-1435998f3db6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-2a932ded-ae2e-4fd3-a003-1435998f3db6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:11:01.630092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:11:01.631277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:11:01.632250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 656/1298] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:11:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:11:01.637225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8157cd5-60c1-437f-ad9f-4a75eda5fc94 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:11:01.655261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:11:01.735919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8157cd5-60c1-437f-ad9f-4a75eda5fc94 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:11:01.771667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb0675-90f8-453a-904f-0d9b372ea40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:11:01.772305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb0675-90f8-453a-904f-0d9b372ea40b None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:01.772594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb0675-90f8-453a-904f-0d9b372ea40b None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:01.772865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb0675-90f8-453a-904f-0d9b372ea40b None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:01.773493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb0675-90f8-453a-904f-0d9b372ea40b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:11:01.773796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb0675-90f8-453a-904f-0d9b372ea40b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:11:01.775198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb0675-90f8-453a-904f-0d9b372ea40b None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:11:01.780259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 643/1299] 10.222.0.71 () {56 vars in 1116 bytes} [Wed Sep 21 12:11:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:11:01.786855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a932ded-ae2e-4fd3-a003-1435998f3db6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:11:01.817297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:11:01.885426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a932ded-ae2e-4fd3-a003-1435998f3db6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:11:01.919144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687b4ff-d099-4761-9c87-c1a5f5825ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:11:01.919775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b4ff-d099-4761-9c87-c1a5f5825ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:01.920056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b4ff-d099-4761-9c87-c1a5f5825ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:01.920328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687b4ff-d099-4761-9c87-c1a5f5825ee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:01.921025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687b4ff-d099-4761-9c87-c1a5f5825ee9 None admin] RBAC: Adding query filter params (interface=public, service_id=a2d8f6aa582643efb19a09ce61e2c1c4, region_id=RegionOne) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:11:01.921357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687b4ff-d099-4761-9c87-c1a5f5825ee9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:11:01.922709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687b4ff-d099-4761-9c87-c1a5f5825ee9 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:11:01.927538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 657/1300] 10.222.0.71 () {56 vars in 1251 bytes} [Wed Sep 21 12:11:01 2022] GET /identity/v3/endpoints?service_id=a2d8f6aa582643efb19a09ce61e2c1c4&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:11:01.933255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb0675-90f8-453a-904f-0d9b372ea40b None admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:11:02.011713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:11:02.031092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb0675-90f8-453a-904f-0d9b372ea40b None admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:11:02.063771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b57d5-318a-4e74-9fcb-027121df9641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:11:02.064403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b57d5-318a-4e74-9fcb-027121df9641 None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:02.064764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b57d5-318a-4e74-9fcb-027121df9641 None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:02.065033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b57d5-318a-4e74-9fcb-027121df9641 None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:02.065732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b57d5-318a-4e74-9fcb-027121df9641 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:11:02.067200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b57d5-318a-4e74-9fcb-027121df9641 None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:11:02.073895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 644/1301] 10.222.0.71 () {56 vars in 1087 bytes} [Wed Sep 21 12:11:01 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:11:04.204665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddeeee-d682-4096-8169-790a15d4783d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:04.205225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddeeee-d682-4096-8169-790a15d4783d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:04.205225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ddeeee-d682-4096-8169-790a15d4783d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:04.206553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 658/1302] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:11:04.211613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd1f86-2e62-430d-ac1a-647f9ad7501c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:04.212030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd1f86-2e62-430d-ac1a-647f9ad7501c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:04.212184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd1f86-2e62-430d-ac1a-647f9ad7501c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:04.229931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-efdd1f86-2e62-430d-ac1a-647f9ad7501c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:11:04.231107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-efdd1f86-2e62-430d-ac1a-647f9ad7501c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:11:04.295987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 645/1303] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:11:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 12:11:04.570278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddeeee-d682-4096-8169-790a15d4783d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:11:04.666625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:11:04.674235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ddeeee-d682-4096-8169-790a15d4783d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:11:04.707348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a4f58-e7e5-4998-aee1-6e232d1452c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:11:04.707996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a4f58-e7e5-4998-aee1-6e232d1452c6 None admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:04.708278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a4f58-e7e5-4998-aee1-6e232d1452c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:04.708553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a4f58-e7e5-4998-aee1-6e232d1452c6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:04.709128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a4f58-e7e5-4998-aee1-6e232d1452c6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:11:04.710490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a4f58-e7e5-4998-aee1-6e232d1452c6 None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application [None req-f13a4f58-e7e5-4998-aee1-6e232d1452c6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 12:11:04.719413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 12:11:04.720643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application with Lock( Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self._enter() Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application return self.creator() Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application created_value = creator( Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: ERROR keystone.server.flask.application Sep 21 12:11:04.721478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 659/1304] 10.222.0.71 () {56 vars in 1117 bytes} [Wed Sep 21 12:11:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 12:11:04.726709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd1f86-2e62-430d-ac1a-647f9ad7501c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:11:04.814049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:11:04.836690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd1f86-2e62-430d-ac1a-647f9ad7501c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:11:04.870096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889d9e3-67c2-4c17-ba7f-dd12087c8c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:11:04.870688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889d9e3-67c2-4c17-ba7f-dd12087c8c8f None admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:04.870962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889d9e3-67c2-4c17-ba7f-dd12087c8c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:04.871242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889d9e3-67c2-4c17-ba7f-dd12087c8c8f None admin] PATH_INFO: `/v3/services` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:04.871895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889d9e3-67c2-4c17-ba7f-dd12087c8c8f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:11:04.872197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889d9e3-67c2-4c17-ba7f-dd12087c8c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:11:04.873553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889d9e3-67c2-4c17-ba7f-dd12087c8c8f None admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:11:04.879068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 646/1305] 10.222.0.71 () {56 vars in 1116 bytes} [Wed Sep 21 12:11:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:11:04.885205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a4f58-e7e5-4998-aee1-6e232d1452c6 None admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:11:04.985389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a4f58-e7e5-4998-aee1-6e232d1452c6 None admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:11:05.017954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00e3af-8288-4594-9462-d9258cd95a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:11:05.018548 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00e3af-8288-4594-9462-d9258cd95a1c None admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:11:05.018827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00e3af-8288-4594-9462-d9258cd95a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:11:05.019097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00e3af-8288-4594-9462-d9258cd95a1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:11:05.019778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00e3af-8288-4594-9462-d9258cd95a1c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:11:05.021143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00e3af-8288-4594-9462-d9258cd95a1c None admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:11:05.040476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 660/1306] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:11:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:11:14.875568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=30 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:11:15.036240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:09.765669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5e223-f472-4517-b705-480cb81aa295 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:09.766125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5e223-f472-4517-b705-480cb81aa295 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:09.766217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5e223-f472-4517-b705-480cb81aa295 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:09.767677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 647/1307] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:09.773175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea3cf4-ba13-42f8-b4ba-5979ab0f4cc8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:09.773580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea3cf4-ba13-42f8-b4ba-5979ab0f4cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:09.773802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea3cf4-ba13-42f8-b4ba-5979ab0f4cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:09.878840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-11ea3cf4-ba13-42f8-b4ba-5979ab0f4cc8 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:09.880397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-11ea3cf4-ba13-42f8-b4ba-5979ab0f4cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:10.004034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 661/1308] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:10.298819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c7e5c-0406-4f57-b99f-7e517916720a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:10.299106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c7e5c-0406-4f57-b99f-7e517916720a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:10.299521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c7e5c-0406-4f57-b99f-7e517916720a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:10.301104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 648/1309] 10.222.0.71 () {54 vars in 963 bytes} [Wed Sep 21 12:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:10.307224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab4ac5-124c-40a7-8183-17bbd802dfc6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:10.307518 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab4ac5-124c-40a7-8183-17bbd802dfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:10.307824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab4ac5-124c-40a7-8183-17bbd802dfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:10.391864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-80ab4ac5-124c-40a7-8183-17bbd802dfc6 None None] MFA Rules not processed for user `cd66d5d9989449729cec7e60cdaf0949`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:10.393086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-80ab4ac5-124c-40a7-8183-17bbd802dfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:10.415355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:10.476409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 662/1310] 10.222.0.71 () {58 vars in 1060 bytes} [Wed Sep 21 12:12:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:10.485234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c7e5c-0406-4f57-b99f-7e517916720a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:10.486975 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0a4c7e5c-0406-4f57-b99f-7e517916720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:10.607008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c7e5c-0406-4f57-b99f-7e517916720a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:10.686204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16f863-07a8-412b-9c2e-50d34a04b864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:10.686814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16f863-07a8-412b-9c2e-50d34a04b864 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:10.687097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16f863-07a8-412b-9c2e-50d34a04b864 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:10.687379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16f863-07a8-412b-9c2e-50d34a04b864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:10.688983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ec16f863-07a8-412b-9c2e-50d34a04b864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:10.745789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:10.806062 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16f863-07a8-412b-9c2e-50d34a04b864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:10.807498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16f863-07a8-412b-9c2e-50d34a04b864 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:10.827800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 649/1311] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:13.351724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b500f11-ad2a-4fa1-a05e-09a9c5cf0f6e None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:13.352290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b500f11-ad2a-4fa1-a05e-09a9c5cf0f6e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:13.352290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b500f11-ad2a-4fa1-a05e-09a9c5cf0f6e None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:13.353774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 663/1312] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:13.359455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85b232-af17-4f43-afb6-9d9818615795 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:13.359929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85b232-af17-4f43-afb6-9d9818615795 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:13.360143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85b232-af17-4f43-afb6-9d9818615795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:13.447002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0c85b232-af17-4f43-afb6-9d9818615795 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:13.448274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0c85b232-af17-4f43-afb6-9d9818615795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:13.546217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 650/1313] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:13.834912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b500f11-ad2a-4fa1-a05e-09a9c5cf0f6e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:13.836509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9b500f11-ad2a-4fa1-a05e-09a9c5cf0f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:13.943579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b500f11-ad2a-4fa1-a05e-09a9c5cf0f6e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:14.014765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:14.017137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb230354-868b-4b0b-8e8d-d3381314820a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:14.017946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb230354-868b-4b0b-8e8d-d3381314820a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:14.018288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb230354-868b-4b0b-8e8d-d3381314820a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:14.018752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb230354-868b-4b0b-8e8d-d3381314820a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:14.020656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eb230354-868b-4b0b-8e8d-d3381314820a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:14.140649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb230354-868b-4b0b-8e8d-d3381314820a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:14.142082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb230354-868b-4b0b-8e8d-d3381314820a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:14.159783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 664/1314] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:16.608146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13730f3e-6992-4719-8bd4-6fefce9643e8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:16.608679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13730f3e-6992-4719-8bd4-6fefce9643e8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:16.608679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13730f3e-6992-4719-8bd4-6fefce9643e8 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:16.610011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 651/1315] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:16.615401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82afc78-6086-4001-b242-05040ceae114 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:16.615865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82afc78-6086-4001-b242-05040ceae114 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:16.615970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82afc78-6086-4001-b242-05040ceae114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:16.681700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b82afc78-6086-4001-b242-05040ceae114 None None] MFA Rules not processed for user `2040a306bf664576a535972455e70a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:16.682962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b82afc78-6086-4001-b242-05040ceae114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:16.734487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:16.780277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 665/1316] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:17.078638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006de27-636c-421d-8476-414b1bcbb699 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:17.078924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006de27-636c-421d-8476-414b1bcbb699 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:17.079200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006de27-636c-421d-8476-414b1bcbb699 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:17.080581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 652/1317] 10.222.0.71 () {54 vars in 963 bytes} [Wed Sep 21 12:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:17.088009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8054e7a-5792-414e-8d1c-d47edda573ed None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:17.088296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8054e7a-5792-414e-8d1c-d47edda573ed None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:17.088570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8054e7a-5792-414e-8d1c-d47edda573ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:17.167081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f8054e7a-5792-414e-8d1c-d47edda573ed None None] MFA Rules not processed for user `cd66d5d9989449729cec7e60cdaf0949`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:17.168341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f8054e7a-5792-414e-8d1c-d47edda573ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:17.242858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 666/1318] 10.222.0.71 () {58 vars in 1060 bytes} [Wed Sep 21 12:12:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:17.252555 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006de27-636c-421d-8476-414b1bcbb699 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:17.254121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8006de27-636c-421d-8476-414b1bcbb699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:17.264651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:17.360143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006de27-636c-421d-8476-414b1bcbb699 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:17.436517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd145df-3c8c-49b7-b78e-4c59e79d0125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:17.437120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd145df-3c8c-49b7-b78e-4c59e79d0125 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:17.437393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd145df-3c8c-49b7-b78e-4c59e79d0125 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:17.437669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd145df-3c8c-49b7-b78e-4c59e79d0125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:17.439251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7cd145df-3c8c-49b7-b78e-4c59e79d0125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:17.501123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:17.547617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd145df-3c8c-49b7-b78e-4c59e79d0125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:17.549086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd145df-3c8c-49b7-b78e-4c59e79d0125 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:17.567563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 653/1319] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:17 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:20.627968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baba84b-84f5-4e63-8dd3-491cee3c9ec4 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:20.628517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baba84b-84f5-4e63-8dd3-491cee3c9ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:20.628614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baba84b-84f5-4e63-8dd3-491cee3c9ec4 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:20.630097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 667/1320] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:20.635231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eeb38-558f-41b5-967e-65a2c6ca0485 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:20.635650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eeb38-558f-41b5-967e-65a2c6ca0485 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:20.635801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eeb38-558f-41b5-967e-65a2c6ca0485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:20.702938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-897eeb38-558f-41b5-967e-65a2c6ca0485 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:20.704199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-897eeb38-558f-41b5-967e-65a2c6ca0485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:20.799274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 654/1321] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:21.086464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baba84b-84f5-4e63-8dd3-491cee3c9ec4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:21.088315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9baba84b-84f5-4e63-8dd3-491cee3c9ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:21.140892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:21.195008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baba84b-84f5-4e63-8dd3-491cee3c9ec4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:21.267483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d2da1-0cb3-4174-bc70-b4966ec1338e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:21.268110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2da1-0cb3-4174-bc70-b4966ec1338e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:21.268387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2da1-0cb3-4174-bc70-b4966ec1338e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:21.268651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2da1-0cb3-4174-bc70-b4966ec1338e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:21.270242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f08d2da1-0cb3-4174-bc70-b4966ec1338e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:21.369651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:21.383686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d2da1-0cb3-4174-bc70-b4966ec1338e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:21.385164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d2da1-0cb3-4174-bc70-b4966ec1338e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:21.405455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 668/1322] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:24.309594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b94dd-292f-4795-906b-7719baff14bf None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:24.310133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b94dd-292f-4795-906b-7719baff14bf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:24.310133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b94dd-292f-4795-906b-7719baff14bf None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:24.311495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 655/1323] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:24.316895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d0722-ea7e-4e54-a9a2-4b9ec03c0bfd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:24.317310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d0722-ea7e-4e54-a9a2-4b9ec03c0bfd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:24.317447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d0722-ea7e-4e54-a9a2-4b9ec03c0bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:24.394433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0e2d0722-ea7e-4e54-a9a2-4b9ec03c0bfd None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:24.396077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e2d0722-ea7e-4e54-a9a2-4b9ec03c0bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:24.508782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 669/1324] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:24.797998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b94dd-292f-4795-906b-7719baff14bf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:24.800004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d78b94dd-292f-4795-906b-7719baff14bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:24.819744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:24.907305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b94dd-292f-4795-906b-7719baff14bf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:24.981102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b61fb6-79b2-498e-967a-5107ad321b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:24.981712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b61fb6-79b2-498e-967a-5107ad321b5a service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:24.981993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b61fb6-79b2-498e-967a-5107ad321b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:24.982265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b61fb6-79b2-498e-967a-5107ad321b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:24.983946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-58b61fb6-79b2-498e-967a-5107ad321b5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:25.053914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:25.094636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b61fb6-79b2-498e-967a-5107ad321b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:25.096116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b61fb6-79b2-498e-967a-5107ad321b5a service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:25.114477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 656/1325] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:27.344814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad900cab-a510-4b1d-b749-e8b535277474 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:27.345434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad900cab-a510-4b1d-b749-e8b535277474 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:27.345434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad900cab-a510-4b1d-b749-e8b535277474 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:27.346701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 670/1326] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:27.352423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ed40f-0b03-43ba-8e5a-f42d22403744 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:27.352833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ed40f-0b03-43ba-8e5a-f42d22403744 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:27.353152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ed40f-0b03-43ba-8e5a-f42d22403744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:27.424845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a84ed40f-0b03-43ba-8e5a-f42d22403744 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:27.426054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a84ed40f-0b03-43ba-8e5a-f42d22403744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:27.538606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 657/1327] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:27.815775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad900cab-a510-4b1d-b749-e8b535277474 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:27.817389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ad900cab-a510-4b1d-b749-e8b535277474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:27.882562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:27.929725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad900cab-a510-4b1d-b749-e8b535277474 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:28.006107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bde6f9-ffb2-4027-a574-774a3aa22857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:28.006720 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde6f9-ffb2-4027-a574-774a3aa22857 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:28.006999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde6f9-ffb2-4027-a574-774a3aa22857 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:28.007272 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde6f9-ffb2-4027-a574-774a3aa22857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:28.008900 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-18bde6f9-ffb2-4027-a574-774a3aa22857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:28.118233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:28.121460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bde6f9-ffb2-4027-a574-774a3aa22857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:28.123045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bde6f9-ffb2-4027-a574-774a3aa22857 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:28.142822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 671/1328] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:30.423661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1682fe6-6068-4be7-9f77-2d10a4969a05 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:30.424331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1682fe6-6068-4be7-9f77-2d10a4969a05 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:30.424331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1682fe6-6068-4be7-9f77-2d10a4969a05 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:30.425511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 658/1329] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:30.430765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066f769-6f15-4434-82fb-5b06dbecde60 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:30.431163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066f769-6f15-4434-82fb-5b06dbecde60 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:30.431322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066f769-6f15-4434-82fb-5b06dbecde60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:30.500455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2066f769-6f15-4434-82fb-5b06dbecde60 None None] MFA Rules not processed for user `2040a306bf664576a535972455e70a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:30.501691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2066f769-6f15-4434-82fb-5b06dbecde60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:30.605846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 672/1330] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:30.889166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1682fe6-6068-4be7-9f77-2d10a4969a05 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:30.891161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b1682fe6-6068-4be7-9f77-2d10a4969a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:30.925648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:31.015862 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1682fe6-6068-4be7-9f77-2d10a4969a05 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:31.096265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666032e-62d7-42fe-8182-f8949701095b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:31.096875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666032e-62d7-42fe-8182-f8949701095b service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:31.097152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666032e-62d7-42fe-8182-f8949701095b service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:31.097418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666032e-62d7-42fe-8182-f8949701095b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:31.099017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8666032e-62d7-42fe-8182-f8949701095b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:31.189450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:31.228255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666032e-62d7-42fe-8182-f8949701095b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:31.230236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666032e-62d7-42fe-8182-f8949701095b service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:31.248931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 659/1331] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:33.703676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b211202-13d3-44e2-ab50-8d1e5cd0b8b1 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:33.704338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b211202-13d3-44e2-ab50-8d1e5cd0b8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:33.704338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b211202-13d3-44e2-ab50-8d1e5cd0b8b1 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:33.705609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 673/1332] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:33.710804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cc29a-7adc-40f0-a034-3e4e4633832c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:33.711209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cc29a-7adc-40f0-a034-3e4e4633832c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:33.711364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cc29a-7adc-40f0-a034-3e4e4633832c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:33.778879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-897cc29a-7adc-40f0-a034-3e4e4633832c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:33.780134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-897cc29a-7adc-40f0-a034-3e4e4633832c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:33.877162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 660/1333] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:34.156023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b211202-13d3-44e2-ab50-8d1e5cd0b8b1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:34.157559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0b211202-13d3-44e2-ab50-8d1e5cd0b8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:34.225779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:34.258983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b211202-13d3-44e2-ab50-8d1e5cd0b8b1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:34.331552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456d517-11a4-44bf-a66a-5fa625dcd347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:34.332186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456d517-11a4-44bf-a66a-5fa625dcd347 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:34.332467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456d517-11a4-44bf-a66a-5fa625dcd347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:34.332742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456d517-11a4-44bf-a66a-5fa625dcd347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:34.334345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8456d517-11a4-44bf-a66a-5fa625dcd347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:34.442281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456d517-11a4-44bf-a66a-5fa625dcd347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:34.443631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456d517-11a4-44bf-a66a-5fa625dcd347 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:34.453851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:34.462535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 674/1334] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:36.671443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f38cd4-41a6-4f5d-bafa-2466e3e3ebb2 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:36.672008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f38cd4-41a6-4f5d-bafa-2466e3e3ebb2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:36.672008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f38cd4-41a6-4f5d-bafa-2466e3e3ebb2 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:36.673281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 661/1335] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:36.678779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d5579-dfc7-4be3-95b3-e635b5823574 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:36.679231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d5579-dfc7-4be3-95b3-e635b5823574 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:36.679338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d5579-dfc7-4be3-95b3-e635b5823574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:36.745287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-615d5579-dfc7-4be3-95b3-e635b5823574 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:36.746512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-615d5579-dfc7-4be3-95b3-e635b5823574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:36.845116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 675/1336] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:37.131323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f38cd4-41a6-4f5d-bafa-2466e3e3ebb2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:37.132972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-87f38cd4-41a6-4f5d-bafa-2466e3e3ebb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:37.173753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:37.236481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f38cd4-41a6-4f5d-bafa-2466e3e3ebb2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:37.310310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3aefa-2e8e-4393-b006-c808b4226fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:37.310905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3aefa-2e8e-4393-b006-c808b4226fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:37.311179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3aefa-2e8e-4393-b006-c808b4226fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:37.311443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3aefa-2e8e-4393-b006-c808b4226fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:37.313075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-25d3aefa-2e8e-4393-b006-c808b4226fb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:37.394104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:37.420341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3aefa-2e8e-4393-b006-c808b4226fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:37.421786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3aefa-2e8e-4393-b006-c808b4226fb1 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:37.440125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 662/1337] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:39.966411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd624d1-da73-493f-b709-753d60f0c393 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:39.966966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd624d1-da73-493f-b709-753d60f0c393 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:39.967076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd624d1-da73-493f-b709-753d60f0c393 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:39.968405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 676/1338] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:39.973662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e0a85-4df1-457f-962b-0b047eb1d6ff None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:39.974075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e0a85-4df1-457f-962b-0b047eb1d6ff None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:39.974209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e0a85-4df1-457f-962b-0b047eb1d6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:40.039615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-586e0a85-4df1-457f-962b-0b047eb1d6ff None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:40.040771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-586e0a85-4df1-457f-962b-0b047eb1d6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:40.134123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 663/1339] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:12:40.415562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd624d1-da73-493f-b709-753d60f0c393 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:40.417129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bcd624d1-da73-493f-b709-753d60f0c393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:40.491843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:40.520249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd624d1-da73-493f-b709-753d60f0c393 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:40.595488 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467cccb-8110-4ef2-b630-f83ae70d1b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:40.596130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467cccb-8110-4ef2-b630-f83ae70d1b2b service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:40.596439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467cccb-8110-4ef2-b630-f83ae70d1b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:40.596711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467cccb-8110-4ef2-b630-f83ae70d1b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:40.598309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f467cccb-8110-4ef2-b630-f83ae70d1b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:40.706578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467cccb-8110-4ef2-b630-f83ae70d1b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:40.707959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467cccb-8110-4ef2-b630-f83ae70d1b2b service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:40.725840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 677/1340] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:44.885748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e14eb8-898f-4a9b-96a6-5f0adf98e0ea None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:44.886812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e14eb8-898f-4a9b-96a6-5f0adf98e0ea None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:44.886890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e14eb8-898f-4a9b-96a6-5f0adf98e0ea None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:44.888440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 664/1341] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:44.895161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149e026-fd33-40ee-89fb-e2e63904f0d7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:44.895669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149e026-fd33-40ee-89fb-e2e63904f0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:44.895669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149e026-fd33-40ee-89fb-e2e63904f0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:44.901084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:44.967900 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3149e026-fd33-40ee-89fb-e2e63904f0d7 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:44.969032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3149e026-fd33-40ee-89fb-e2e63904f0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:45.081763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 678/1342] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:12:45.365717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e14eb8-898f-4a9b-96a6-5f0adf98e0ea None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:45.367258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c3e14eb8-898f-4a9b-96a6-5f0adf98e0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:45.417693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:45.476036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e14eb8-898f-4a9b-96a6-5f0adf98e0ea None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:45.551799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a1432-30d0-49ba-b99c-ed434d8a1f33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:45.552403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a1432-30d0-49ba-b99c-ed434d8a1f33 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:45.552680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a1432-30d0-49ba-b99c-ed434d8a1f33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:45.552949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a1432-30d0-49ba-b99c-ed434d8a1f33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:45.554547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-817a1432-30d0-49ba-b99c-ed434d8a1f33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:45.648551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:45.665077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a1432-30d0-49ba-b99c-ed434d8a1f33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:45.666512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a1432-30d0-49ba-b99c-ed434d8a1f33 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:45.685830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 665/1343] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:48.650585 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba05cf-7c71-488e-b31a-99194443ed55 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:48.651131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba05cf-7c71-488e-b31a-99194443ed55 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:48.651131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba05cf-7c71-488e-b31a-99194443ed55 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:48.652542 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 679/1344] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:48.657625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c6ad4-8e01-49a7-8804-5dbcdf07ed30 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:48.658105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c6ad4-8e01-49a7-8804-5dbcdf07ed30 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:48.658308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c6ad4-8e01-49a7-8804-5dbcdf07ed30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:48.746340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bf9c6ad4-8e01-49a7-8804-5dbcdf07ed30 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:48.747600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bf9c6ad4-8e01-49a7-8804-5dbcdf07ed30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:48.857733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 666/1345] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:49.154274 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba05cf-7c71-488e-b31a-99194443ed55 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:49.155946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22ba05cf-7c71-488e-b31a-99194443ed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:49.250728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:49.269853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba05cf-7c71-488e-b31a-99194443ed55 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:49.354231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a2948-df24-4ce5-8b2a-8fd2ab2b03f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:49.355416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a2948-df24-4ce5-8b2a-8fd2ab2b03f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:49.355694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a2948-df24-4ce5-8b2a-8fd2ab2b03f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:49.355989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a2948-df24-4ce5-8b2a-8fd2ab2b03f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:49.357613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fa5a2948-df24-4ce5-8b2a-8fd2ab2b03f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:49.477388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a2948-df24-4ce5-8b2a-8fd2ab2b03f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:49.478841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a2948-df24-4ce5-8b2a-8fd2ab2b03f1 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:49.519997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 680/1346] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:12:53.878705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e41439-7e3d-4926-9b8b-14d8cde6670e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:53.879208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e41439-7e3d-4926-9b8b-14d8cde6670e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:53.879317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e41439-7e3d-4926-9b8b-14d8cde6670e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:53.880638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 667/1347] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:53.887514 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9fa7-5062-4304-be58-e08730b60cc8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:53.888072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9fa7-5062-4304-be58-e08730b60cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:53.888433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a9fa7-5062-4304-be58-e08730b60cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:53.911479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:53.964655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-876a9fa7-5062-4304-be58-e08730b60cc8 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:53.965863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-876a9fa7-5062-4304-be58-e08730b60cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:54.064056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 681/1348] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:12:54.348667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e41439-7e3d-4926-9b8b-14d8cde6670e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:12:54.350202 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c2e41439-7e3d-4926-9b8b-14d8cde6670e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:54.408636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:54.453909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e41439-7e3d-4926-9b8b-14d8cde6670e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:12:54.528331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647cb752-8c3e-4f98-bb8a-7f952c4c91f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:12:54.528930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cb752-8c3e-4f98-bb8a-7f952c4c91f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:54.529207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cb752-8c3e-4f98-bb8a-7f952c4c91f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:54.529478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647cb752-8c3e-4f98-bb8a-7f952c4c91f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:54.531079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-647cb752-8c3e-4f98-bb8a-7f952c4c91f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:54.635015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:12:54.639696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647cb752-8c3e-4f98-bb8a-7f952c4c91f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:12:54.641153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647cb752-8c3e-4f98-bb8a-7f952c4c91f6 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:12:54.659392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 668/1349] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:12:59.637893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57192042-5549-4b87-b1e6-684ca3a6476b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:59.638371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57192042-5549-4b87-b1e6-684ca3a6476b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:59.638461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57192042-5549-4b87-b1e6-684ca3a6476b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:59.639934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 682/1350] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:12:59.645569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bfb0e-2a9f-4f2a-971d-2a4785f6f383 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:12:59.646278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bfb0e-2a9f-4f2a-971d-2a4785f6f383 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:12:59.646278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bfb0e-2a9f-4f2a-971d-2a4785f6f383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:12:59.754740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-436bfb0e-2a9f-4f2a-971d-2a4785f6f383 None None] MFA Rules not processed for user `eac29fd43f1f4bbd987950b90385200b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:12:59.754740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-436bfb0e-2a9f-4f2a-971d-2a4785f6f383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:12:59.923594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 669/1351] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:12:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:04.062536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=59 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:04.281726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce765d-4d57-446b-8476-b1305d3dbf09 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:04.282268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce765d-4d57-446b-8476-b1305d3dbf09 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:04.282521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce765d-4d57-446b-8476-b1305d3dbf09 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:04.284184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 683/1352] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:04.289260 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84e7b-d005-4131-a620-5ef534d00e6e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:04.289717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84e7b-d005-4131-a620-5ef534d00e6e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:04.289800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb84e7b-d005-4131-a620-5ef534d00e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:04.350848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:04.381993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1cb84e7b-d005-4131-a620-5ef534d00e6e None None] MFA Rules not processed for user `eac29fd43f1f4bbd987950b90385200b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:04.383497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1cb84e7b-d005-4131-a620-5ef534d00e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:04.506191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 670/1353] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:13:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:05.636951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa30e23-fb93-4751-b79d-9c3e80c734cc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:05.637486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa30e23-fb93-4751-b79d-9c3e80c734cc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:05.637486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa30e23-fb93-4751-b79d-9c3e80c734cc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:05.638819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 684/1354] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:13:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:05.644564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578aef2-0862-4355-b87e-8f218fa8603d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:05.645021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578aef2-0862-4355-b87e-8f218fa8603d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:05.645270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578aef2-0862-4355-b87e-8f218fa8603d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:05.721583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8578aef2-0862-4355-b87e-8f218fa8603d None None] MFA Rules not processed for user `eac29fd43f1f4bbd987950b90385200b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:05.722773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8578aef2-0862-4355-b87e-8f218fa8603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:05.770301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:05.842451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 671/1355] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:13:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:13:09.403221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561dfdd-4261-4931-bb27-71d4b9cc272a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:09.404129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561dfdd-4261-4931-bb27-71d4b9cc272a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:09.404890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561dfdd-4261-4931-bb27-71d4b9cc272a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:09.406579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 685/1356] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:09.412262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ff16a-fbb2-44ab-a646-da8e808b1948 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:09.412738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ff16a-fbb2-44ab-a646-da8e808b1948 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:09.412844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ff16a-fbb2-44ab-a646-da8e808b1948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:09.491248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a65ff16a-fbb2-44ab-a646-da8e808b1948 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:09.492490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a65ff16a-fbb2-44ab-a646-da8e808b1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:09.599847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 672/1357] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:09.909883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99f21e-863b-4ca6-b3d0-fe264423f7e7 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:09.910376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99f21e-863b-4ca6-b3d0-fe264423f7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:09.910765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99f21e-863b-4ca6-b3d0-fe264423f7e7 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:09.912464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 686/1358] 10.222.0.71 () {54 vars in 965 bytes} [Wed Sep 21 12:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:09.922847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9d952-ad1d-488a-81a3-3e2991ebc539 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:09.923581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9d952-ad1d-488a-81a3-3e2991ebc539 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:09.924217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9d952-ad1d-488a-81a3-3e2991ebc539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:09.946277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:10.030099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-70c9d952-ad1d-488a-81a3-3e2991ebc539 None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:10.032430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-70c9d952-ad1d-488a-81a3-3e2991ebc539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:10.164192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 673/1359] 10.222.0.71 () {58 vars in 1062 bytes} [Wed Sep 21 12:13:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:10.165288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99f21e-863b-4ca6-b3d0-fe264423f7e7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:10.167375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7f99f21e-863b-4ca6-b3d0-fe264423f7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:10.340749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99f21e-863b-4ca6-b3d0-fe264423f7e7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:10.420402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:10.464697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12990467-f2eb-4134-b3a3-c18706664e14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:10.465302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12990467-f2eb-4134-b3a3-c18706664e14 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:10.465611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12990467-f2eb-4134-b3a3-c18706664e14 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:10.465869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12990467-f2eb-4134-b3a3-c18706664e14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:10.467436 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12990467-f2eb-4134-b3a3-c18706664e14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:10.590424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12990467-f2eb-4134-b3a3-c18706664e14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:10.593149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12990467-f2eb-4134-b3a3-c18706664e14 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:10.621586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 687/1360] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:13.042710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290e956-e514-450f-90cd-6282ce39991a None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:13.043309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290e956-e514-450f-90cd-6282ce39991a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:13.043309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290e956-e514-450f-90cd-6282ce39991a None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:13.044625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 674/1361] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:13.049808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6423b3-4b37-4537-8168-1dbe3216c14f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:13.050296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6423b3-4b37-4537-8168-1dbe3216c14f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:13.050458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6423b3-4b37-4537-8168-1dbe3216c14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:13.120574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0c6423b3-4b37-4537-8168-1dbe3216c14f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:13.121866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0c6423b3-4b37-4537-8168-1dbe3216c14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:13.162052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:13.225831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 688/1362] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:13.528317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290e956-e514-450f-90cd-6282ce39991a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:13.529797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4290e956-e514-450f-90cd-6282ce39991a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:13.614405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:13.717643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290e956-e514-450f-90cd-6282ce39991a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:13.851459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78849fb1-4df2-47eb-9b87-8603f5115aba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:13.852102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78849fb1-4df2-47eb-9b87-8603f5115aba service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:13.852381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78849fb1-4df2-47eb-9b87-8603f5115aba service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:13.852653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78849fb1-4df2-47eb-9b87-8603f5115aba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:13.854843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-78849fb1-4df2-47eb-9b87-8603f5115aba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:13.879466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:13.961676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78849fb1-4df2-47eb-9b87-8603f5115aba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:13.964751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78849fb1-4df2-47eb-9b87-8603f5115aba service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:13.987316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 675/1363] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:16.511974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082b008-84fe-4f87-8c6e-496946e00bad None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:16.512563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082b008-84fe-4f87-8c6e-496946e00bad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:16.512563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082b008-84fe-4f87-8c6e-496946e00bad None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:16.513993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 689/1364] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:16.519163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da072e3-32ba-4e88-b85f-284fb0becac7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:16.519566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da072e3-32ba-4e88-b85f-284fb0becac7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:16.519791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da072e3-32ba-4e88-b85f-284fb0becac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:16.590556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7da072e3-32ba-4e88-b85f-284fb0becac7 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:16.591973 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7da072e3-32ba-4e88-b85f-284fb0becac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:16.693216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:16.699918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 676/1365] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:13:16.992094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca24d7-f0c4-49f9-9700-5a4bc7e2a6f9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:16.992250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca24d7-f0c4-49f9-9700-5a4bc7e2a6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:16.992521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca24d7-f0c4-49f9-9700-5a4bc7e2a6f9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:16.994611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 690/1366] 10.222.0.71 () {54 vars in 965 bytes} [Wed Sep 21 12:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:16.999963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103ce75-c977-438c-8b12-d6153e6aa608 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:17.000355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103ce75-c977-438c-8b12-d6153e6aa608 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:17.000769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103ce75-c977-438c-8b12-d6153e6aa608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:17.081192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5103ce75-c977-438c-8b12-d6153e6aa608 None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:17.082659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5103ce75-c977-438c-8b12-d6153e6aa608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:17.211345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 677/1367] 10.222.0.71 () {58 vars in 1062 bytes} [Wed Sep 21 12:13:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:17.220064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca24d7-f0c4-49f9-9700-5a4bc7e2a6f9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:17.221638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-29ca24d7-f0c4-49f9-9700-5a4bc7e2a6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:17.389805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca24d7-f0c4-49f9-9700-5a4bc7e2a6f9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:17.412086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:17.521746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae55ca-2447-495d-9299-924e95300cc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:17.522597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae55ca-2447-495d-9299-924e95300cc2 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:17.522985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae55ca-2447-495d-9299-924e95300cc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:17.522985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae55ca-2447-495d-9299-924e95300cc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:17.524806 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97ae55ca-2447-495d-9299-924e95300cc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:17.637338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae55ca-2447-495d-9299-924e95300cc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:17.639519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae55ca-2447-495d-9299-924e95300cc2 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:17.659121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 691/1368] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:17.695191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9a3a-10b3-4cb7-ab0b-dbe3dc2087c4 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:17.695802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9a3a-10b3-4cb7-ab0b-dbe3dc2087c4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:17.695901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc9a3a-10b3-4cb7-ab0b-dbe3dc2087c4 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:17.697325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 678/1369] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:17.703650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb37e0-afa5-4a63-8f7a-6865f3cfaf3b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:17.704148 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb37e0-afa5-4a63-8f7a-6865f3cfaf3b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:17.704530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb37e0-afa5-4a63-8f7a-6865f3cfaf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:17.750041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:17.796039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-39eb37e0-afa5-4a63-8f7a-6865f3cfaf3b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:17.797581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-39eb37e0-afa5-4a63-8f7a-6865f3cfaf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:17.922656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 692/1370] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:18.258149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64d8d2-41f5-4fd1-80ad-d65bd745ed25 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:18.258521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64d8d2-41f5-4fd1-80ad-d65bd745ed25 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:18.258732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64d8d2-41f5-4fd1-80ad-d65bd745ed25 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:18.263791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 679/1371] 10.222.0.71 () {54 vars in 963 bytes} [Wed Sep 21 12:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:18.266169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a061e7-effb-4072-8574-591161b66bf0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:18.266571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a061e7-effb-4072-8574-591161b66bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:18.267468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a061e7-effb-4072-8574-591161b66bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:18.353871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-65a061e7-effb-4072-8574-591161b66bf0 None None] MFA Rules not processed for user `5c84b440821446e3a7ce97b2c68f815e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:18.355291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-65a061e7-effb-4072-8574-591161b66bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:18.404516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:18.437013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 693/1372] 10.222.0.71 () {58 vars in 1060 bytes} [Wed Sep 21 12:13:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:18.444761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64d8d2-41f5-4fd1-80ad-d65bd745ed25 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:18.446993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eb64d8d2-41f5-4fd1-80ad-d65bd745ed25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:18.503298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:18.564798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64d8d2-41f5-4fd1-80ad-d65bd745ed25 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:18.656429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545e30c-62a1-43ea-a7d2-0868f93026f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:18.657256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545e30c-62a1-43ea-a7d2-0868f93026f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:18.657669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545e30c-62a1-43ea-a7d2-0868f93026f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:18.658038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545e30c-62a1-43ea-a7d2-0868f93026f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:18.660206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9545e30c-62a1-43ea-a7d2-0868f93026f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:18.792417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:18.810920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545e30c-62a1-43ea-a7d2-0868f93026f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:18.813135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545e30c-62a1-43ea-a7d2-0868f93026f9 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:18.843460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 680/1373] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:13:19.950676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b8146-a1da-40ef-930e-ef3ef3f462fe None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:19.951239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b8146-a1da-40ef-930e-ef3ef3f462fe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:19.951239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b8146-a1da-40ef-930e-ef3ef3f462fe None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:19.952615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 694/1374] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:19.957907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea17352-548d-433b-ae3c-33ac0f8fabb5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:19.958433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea17352-548d-433b-ae3c-33ac0f8fabb5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:19.958433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea17352-548d-433b-ae3c-33ac0f8fabb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:20.025009 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3ea17352-548d-433b-ae3c-33ac0f8fabb5 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:20.026144 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3ea17352-548d-433b-ae3c-33ac0f8fabb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:20.130656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 681/1375] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:20.424873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b8146-a1da-40ef-930e-ef3ef3f462fe None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:20.426667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fa6b8146-a1da-40ef-930e-ef3ef3f462fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:20.586571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b8146-a1da-40ef-930e-ef3ef3f462fe None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:20.642791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:20.708249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86132990-aa24-4fce-b945-32fb1650bcd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:20.709123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86132990-aa24-4fce-b945-32fb1650bcd2 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:20.709405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86132990-aa24-4fce-b945-32fb1650bcd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:20.709673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86132990-aa24-4fce-b945-32fb1650bcd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:20.711659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86132990-aa24-4fce-b945-32fb1650bcd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:20.810451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86132990-aa24-4fce-b945-32fb1650bcd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:20.812108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86132990-aa24-4fce-b945-32fb1650bcd2 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:20.831235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 695/1376] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:13:23.082489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf048a3-a7d2-4efb-8ea0-425379928416 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:23.083017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf048a3-a7d2-4efb-8ea0-425379928416 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:23.083017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf048a3-a7d2-4efb-8ea0-425379928416 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:23.084585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 682/1377] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:23.090908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5765e5-2b6f-4bba-b4f0-4c612a30cfde None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:23.091382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5765e5-2b6f-4bba-b4f0-4c612a30cfde None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:23.091674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5765e5-2b6f-4bba-b4f0-4c612a30cfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:23.162706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4c5765e5-2b6f-4bba-b4f0-4c612a30cfde None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:23.164169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4c5765e5-2b6f-4bba-b4f0-4c612a30cfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:23.173124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:23.276519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 696/1378] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:23.567174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf048a3-a7d2-4efb-8ea0-425379928416 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:23.568706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6cf048a3-a7d2-4efb-8ea0-425379928416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:23.633194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:23.722944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf048a3-a7d2-4efb-8ea0-425379928416 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:23.843572 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cd801-4139-4499-9366-ce358f398143 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:23.844287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cd801-4139-4499-9366-ce358f398143 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:23.844631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cd801-4139-4499-9366-ce358f398143 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:23.844966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cd801-4139-4499-9366-ce358f398143 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:23.846633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-729cd801-4139-4499-9366-ce358f398143 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:23.866871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:23.943326 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cd801-4139-4499-9366-ce358f398143 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:23.944949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cd801-4139-4499-9366-ce358f398143 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:23.966143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 683/1379] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:26.205967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdd698-52ed-4fd4-b227-001e00b20d3d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:26.206451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdd698-52ed-4fd4-b227-001e00b20d3d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:26.206708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdd698-52ed-4fd4-b227-001e00b20d3d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:26.208335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 697/1380] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:26.213998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc996d87-1a4b-4e64-976b-ce5daf741bc3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:26.214391 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc996d87-1a4b-4e64-976b-ce5daf741bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:26.214586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc996d87-1a4b-4e64-976b-ce5daf741bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:26.287558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bc996d87-1a4b-4e64-976b-ce5daf741bc3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:26.289023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bc996d87-1a4b-4e64-976b-ce5daf741bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:26.404271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:26.411152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 684/1381] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:26.770366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdd698-52ed-4fd4-b227-001e00b20d3d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:26.771988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a7cdd698-52ed-4fd4-b227-001e00b20d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:26.937878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:26.950992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdd698-52ed-4fd4-b227-001e00b20d3d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:27.076838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f506dc3-eaf1-40e1-bb62-ec29736285c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:27.077467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f506dc3-eaf1-40e1-bb62-ec29736285c8 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:27.077752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f506dc3-eaf1-40e1-bb62-ec29736285c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:27.078031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f506dc3-eaf1-40e1-bb62-ec29736285c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:27.081670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5f506dc3-eaf1-40e1-bb62-ec29736285c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:27.180502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f506dc3-eaf1-40e1-bb62-ec29736285c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:27.182107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f506dc3-eaf1-40e1-bb62-ec29736285c8 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:27.192856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:27.201690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 698/1382] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:29.427923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd81a2c-728d-42c3-a12f-c051fe09fd64 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:29.428624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd81a2c-728d-42c3-a12f-c051fe09fd64 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:29.428624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd81a2c-728d-42c3-a12f-c051fe09fd64 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:29.429821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 685/1383] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:13:29.436853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dbf61-e1e5-48c3-b272-5a03f962a8fc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:29.437317 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dbf61-e1e5-48c3-b272-5a03f962a8fc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:29.437615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dbf61-e1e5-48c3-b272-5a03f962a8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:29.529221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9b2dbf61-e1e5-48c3-b272-5a03f962a8fc None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:29.530394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9b2dbf61-e1e5-48c3-b272-5a03f962a8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:29.641114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 699/1384] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:29.928642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd81a2c-728d-42c3-a12f-c051fe09fd64 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:29.930868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-efd81a2c-728d-42c3-a12f-c051fe09fd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:30.106988 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd81a2c-728d-42c3-a12f-c051fe09fd64 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:30.177743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:30.226133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd4a9d-6c33-4e70-ac17-c30c4ddb59a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:30.226741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd4a9d-6c33-4e70-ac17-c30c4ddb59a7 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:30.227209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd4a9d-6c33-4e70-ac17-c30c4ddb59a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:30.227348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd4a9d-6c33-4e70-ac17-c30c4ddb59a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:30.229081 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0fdd4a9d-6c33-4e70-ac17-c30c4ddb59a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:30.320158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd4a9d-6c33-4e70-ac17-c30c4ddb59a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:30.322016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd4a9d-6c33-4e70-ac17-c30c4ddb59a7 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:30.344577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 686/1385] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:32.576505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c9941-f554-4dac-999c-ace9436fd176 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:32.577125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c9941-f554-4dac-999c-ace9436fd176 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:32.577125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c9941-f554-4dac-999c-ace9436fd176 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:32.578402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 700/1386] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:32.583873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e50498-0213-4e39-852c-dc4a7ff5ad8e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:32.584307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e50498-0213-4e39-852c-dc4a7ff5ad8e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:32.584419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e50498-0213-4e39-852c-dc4a7ff5ad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:32.652228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b8e50498-0213-4e39-852c-dc4a7ff5ad8e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:32.653453 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b8e50498-0213-4e39-852c-dc4a7ff5ad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:32.687285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:32.756638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 687/1387] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:33.045236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c9941-f554-4dac-999c-ace9436fd176 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:33.046830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4a8c9941-f554-4dac-999c-ace9436fd176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:33.124376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:33.198908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c9941-f554-4dac-999c-ace9436fd176 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:33.315469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a57f7-6b4e-4156-b855-69e7adc25dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:33.316111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a57f7-6b4e-4156-b855-69e7adc25dd3 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:33.316381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a57f7-6b4e-4156-b855-69e7adc25dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:33.316692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a57f7-6b4e-4156-b855-69e7adc25dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:33.318371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-113a57f7-6b4e-4156-b855-69e7adc25dd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:33.358911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:33.411220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a57f7-6b4e-4156-b855-69e7adc25dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:33.412843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a57f7-6b4e-4156-b855-69e7adc25dd3 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:33.432307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 701/1388] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:34.142076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d9dd7-6222-44a0-bcaf-d3214f992a8e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:34.142618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d9dd7-6222-44a0-bcaf-d3214f992a8e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:34.142738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d9dd7-6222-44a0-bcaf-d3214f992a8e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:34.144105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 688/1389] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:34.150112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977178f-9b0a-400b-9e8c-96fb8678a464 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:34.150613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977178f-9b0a-400b-9e8c-96fb8678a464 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:34.150712 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977178f-9b0a-400b-9e8c-96fb8678a464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:34.218629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6977178f-9b0a-400b-9e8c-96fb8678a464 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:34.219870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6977178f-9b0a-400b-9e8c-96fb8678a464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:34.317228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 702/1390] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:34.619407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0a96-0be8-4843-91c5-2d39a235f984 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:34.619785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0a96-0be8-4843-91c5-2d39a235f984 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:34.620145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0a96-0be8-4843-91c5-2d39a235f984 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:34.622182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 689/1391] 10.222.0.71 () {54 vars in 962 bytes} [Wed Sep 21 12:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:34.627094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d78fe5-e38b-498b-8fb3-355fda7f14f0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:34.627395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d78fe5-e38b-498b-8fb3-355fda7f14f0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:34.627661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d78fe5-e38b-498b-8fb3-355fda7f14f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:34.644314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:34.706582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a1d78fe5-e38b-498b-8fb3-355fda7f14f0 None None] MFA Rules not processed for user `56a94ccca15c462c9e7a5e88b907f420`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:34.707799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a1d78fe5-e38b-498b-8fb3-355fda7f14f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:34.788341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 703/1392] 10.222.0.71 () {58 vars in 1059 bytes} [Wed Sep 21 12:13:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:34.797333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce0a96-0be8-4843-91c5-2d39a235f984 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:34.798828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0bce0a96-0be8-4843-91c5-2d39a235f984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:34.911264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce0a96-0be8-4843-91c5-2d39a235f984 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:34.971007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:34.987491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf116bf-1b9d-4b44-9ab1-5f0d04dc5ab5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56a94ccca15c462c9e7a5e88b907f420', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:34.988116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf116bf-1b9d-4b44-9ab1-5f0d04dc5ab5 service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:34.988396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf116bf-1b9d-4b44-9ab1-5f0d04dc5ab5 service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:34.988673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf116bf-1b9d-4b44-9ab1-5f0d04dc5ab5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:34.990256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eaf116bf-1b9d-4b44-9ab1-5f0d04dc5ab5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:35.120948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf116bf-1b9d-4b44-9ab1-5f0d04dc5ab5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:35.123462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf116bf-1b9d-4b44-9ab1-5f0d04dc5ab5 service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:35.142324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 690/1393] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:35.648357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c154978-b0b7-492d-9e72-f8a2c27c6f20 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:35.649519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c154978-b0b7-492d-9e72-f8a2c27c6f20 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:35.649641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c154978-b0b7-492d-9e72-f8a2c27c6f20 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:35.650977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 704/1394] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:35.656566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512584-7e5e-4c6a-b850-b0cfeec8f53f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:35.656966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512584-7e5e-4c6a-b850-b0cfeec8f53f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:35.657322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512584-7e5e-4c6a-b850-b0cfeec8f53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:35.730045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-66512584-7e5e-4c6a-b850-b0cfeec8f53f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:35.731312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-66512584-7e5e-4c6a-b850-b0cfeec8f53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:35.781099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:35.839963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 691/1395] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:35.892043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f709887-064a-417b-83bd-29d8a2963fbe None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:35.892331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f709887-064a-417b-83bd-29d8a2963fbe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:35.892603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f709887-064a-417b-83bd-29d8a2963fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:35.962765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4f709887-064a-417b-83bd-29d8a2963fbe None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:35.964009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f709887-064a-417b-83bd-29d8a2963fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:36.022783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:36.055981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 705/1396] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:13:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:36.098335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b34fa-afd4-4c2b-83fc-09403e098179 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:36.098622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b34fa-afd4-4c2b-83fc-09403e098179 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:36.098894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b34fa-afd4-4c2b-83fc-09403e098179 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:36.100268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 692/1397] 10.222.0.71 () {54 vars in 943 bytes} [Wed Sep 21 12:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:36.106625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284cce6-92a6-42cd-bf29-642eb5c58737 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:36.106939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284cce6-92a6-42cd-bf29-642eb5c58737 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:36.107215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284cce6-92a6-42cd-bf29-642eb5c58737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:36.142448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b34fa-afd4-4c2b-83fc-09403e098179 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:36.144065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b27b34fa-afd4-4c2b-83fc-09403e098179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:36.189956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6284cce6-92a6-42cd-bf29-642eb5c58737 None None] MFA Rules not processed for user `e0bf75c730c946bc9168dd418e7af637`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:36.191253 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6284cce6-92a6-42cd-bf29-642eb5c58737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:36.293807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b34fa-afd4-4c2b-83fc-09403e098179 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:36.302783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 706/1398] 10.222.0.71 () {58 vars in 1040 bytes} [Wed Sep 21 12:13:36 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:36.313919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284cce6-92a6-42cd-bf29-642eb5c58737 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:36.326017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284cce6-92a6-42cd-bf29-642eb5c58737 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:36.332047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:36.354207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:36.423234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59359eff-21f2-4874-b5c4-b8d49773f0b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:36.424031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59359eff-21f2-4874-b5c4-b8d49773f0b8 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:36.424410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59359eff-21f2-4874-b5c4-b8d49773f0b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:36.424788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59359eff-21f2-4874-b5c4-b8d49773f0b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:36.426497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-59359eff-21f2-4874-b5c4-b8d49773f0b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:36.436442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1340be-f9da-4a1c-ba0f-2132062c5814 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:36.437052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1340be-f9da-4a1c-ba0f-2132062c5814 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:36.437358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1340be-f9da-4a1c-ba0f-2132062c5814 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:36.437628 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1340be-f9da-4a1c-ba0f-2132062c5814 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:36.439250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8e1340be-f9da-4a1c-ba0f-2132062c5814 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:36.530458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1340be-f9da-4a1c-ba0f-2132062c5814 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:36.532089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1340be-f9da-4a1c-ba0f-2132062c5814 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:36.536162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59359eff-21f2-4874-b5c4-b8d49773f0b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:36.538483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59359eff-21f2-4874-b5c4-b8d49773f0b8 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:36.540589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 707/1399] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:13:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 12:13:36.566519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 693/1400] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:13:38.813350 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd343e1-d5a3-4962-92c2-1ef34e8ab4ea None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:38.813941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd343e1-d5a3-4962-92c2-1ef34e8ab4ea None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:38.813941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd343e1-d5a3-4962-92c2-1ef34e8ab4ea None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:38.815234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 708/1401] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:38.821187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419292dc-83dc-4ce2-844b-d20d2dc9933b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:38.821711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419292dc-83dc-4ce2-844b-d20d2dc9933b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:38.821711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419292dc-83dc-4ce2-844b-d20d2dc9933b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:38.859461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:38.896670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-419292dc-83dc-4ce2-844b-d20d2dc9933b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:38.897548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-419292dc-83dc-4ce2-844b-d20d2dc9933b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:39.001886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 694/1402] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:39.300929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd343e1-d5a3-4962-92c2-1ef34e8ab4ea None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:39.302557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cfd343e1-d5a3-4962-92c2-1ef34e8ab4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:39.376083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:39.537484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd343e1-d5a3-4962-92c2-1ef34e8ab4ea None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:39.667901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11758a5a-b2b2-4d86-8666-f393ad0f6c8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:39.668502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11758a5a-b2b2-4d86-8666-f393ad0f6c8a service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:39.668776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11758a5a-b2b2-4d86-8666-f393ad0f6c8a service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:39.669043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11758a5a-b2b2-4d86-8666-f393ad0f6c8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:39.670626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-11758a5a-b2b2-4d86-8666-f393ad0f6c8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:39.691687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:39.783815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11758a5a-b2b2-4d86-8666-f393ad0f6c8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:39.786145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11758a5a-b2b2-4d86-8666-f393ad0f6c8a service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:39.805996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 709/1403] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:42.084105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984057a-8929-4735-a9e8-c4a24c043655 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:42.084751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984057a-8929-4735-a9e8-c4a24c043655 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:42.084751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984057a-8929-4735-a9e8-c4a24c043655 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:42.086455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 695/1404] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:42.094010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622f3f-864c-4db6-8cf2-f40acac81eb3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:42.094522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622f3f-864c-4db6-8cf2-f40acac81eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:42.096619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4622f3f-864c-4db6-8cf2-f40acac81eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:42.197912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f4622f3f-864c-4db6-8cf2-f40acac81eb3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:42.199131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f4622f3f-864c-4db6-8cf2-f40acac81eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:42.316480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:42.338091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 710/1405] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:42.624357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984057a-8929-4735-a9e8-c4a24c043655 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:42.625884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0984057a-8929-4735-a9e8-c4a24c043655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:42.741651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:42.797462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984057a-8929-4735-a9e8-c4a24c043655 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:42.931574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d646a-a545-4609-8dbb-56b70124f31d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:42.931574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d646a-a545-4609-8dbb-56b70124f31d service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:42.931574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d646a-a545-4609-8dbb-56b70124f31d service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:42.931574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d646a-a545-4609-8dbb-56b70124f31d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:42.932156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d46d646a-a545-4609-8dbb-56b70124f31d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:43.012557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:43.031543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d646a-a545-4609-8dbb-56b70124f31d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:43.033193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d646a-a545-4609-8dbb-56b70124f31d service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:43.057199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 696/1406] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:45.292549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121bc44-9ed1-4e5f-8786-1d2580167ebd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:45.293074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121bc44-9ed1-4e5f-8786-1d2580167ebd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:45.293074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121bc44-9ed1-4e5f-8786-1d2580167ebd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:45.294462 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 711/1407] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:45.300397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608858e0-34a9-4739-87dd-b939ab839706 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:45.301244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608858e0-34a9-4739-87dd-b939ab839706 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:45.301606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608858e0-34a9-4739-87dd-b939ab839706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:45.375689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-608858e0-34a9-4739-87dd-b939ab839706 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:45.377054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-608858e0-34a9-4739-87dd-b939ab839706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:45.493869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 697/1408] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:45.790725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121bc44-9ed1-4e5f-8786-1d2580167ebd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:13:45.792369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f121bc44-9ed1-4e5f-8786-1d2580167ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:45.969190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121bc44-9ed1-4e5f-8786-1d2580167ebd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:13:46.052135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:46.108610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f173e-38cc-4e25-abff-1e1abc1ea02a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:13:46.109220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f173e-38cc-4e25-abff-1e1abc1ea02a service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:46.109517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f173e-38cc-4e25-abff-1e1abc1ea02a service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:46.109766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f173e-38cc-4e25-abff-1e1abc1ea02a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:46.111352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-397f173e-38cc-4e25-abff-1e1abc1ea02a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:46.215882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f173e-38cc-4e25-abff-1e1abc1ea02a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:13:46.217610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f173e-38cc-4e25-abff-1e1abc1ea02a service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:13:46.238133 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 712/1409] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:13:55.491305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=79 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:56.235442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=64 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:59.275578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49a9db-f41a-4dc1-a9b4-1c43d1ee15ce None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:59.276195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49a9db-f41a-4dc1-a9b4-1c43d1ee15ce None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:59.276195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49a9db-f41a-4dc1-a9b4-1c43d1ee15ce None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:59.277513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 698/1410] 10.222.0.71 () {54 vars in 918 bytes} [Wed Sep 21 12:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:59.282877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dce2c-dafb-4c55-ab20-6127db79451b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:59.283416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dce2c-dafb-4c55-ab20-6127db79451b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:59.283416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dce2c-dafb-4c55-ab20-6127db79451b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:59.359114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ef5dce2c-dafb-4c55-ab20-6127db79451b None None] MFA Rules not processed for user `eac29fd43f1f4bbd987950b90385200b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:59.360336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef5dce2c-dafb-4c55-ab20-6127db79451b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:59.463836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 713/1411] 10.222.0.71 () {58 vars in 1015 bytes} [Wed Sep 21 12:13:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:13:59.478282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49a85a-897e-44e7-b8b4-e053cee7a572 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:59.478567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49a85a-897e-44e7-b8b4-e053cee7a572 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:59.478842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49a85a-897e-44e7-b8b4-e053cee7a572 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:59.480205 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 699/1412] 10.222.0.71 () {54 vars in 918 bytes} [Wed Sep 21 12:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:13:59.485124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00905ade-fadc-49f9-9ea6-0a1b8937ab5a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:13:59.485412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00905ade-fadc-49f9-9ea6-0a1b8937ab5a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:13:59.485688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00905ade-fadc-49f9-9ea6-0a1b8937ab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:13:59.559454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:13:59.562566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-00905ade-fadc-49f9-9ea6-0a1b8937ab5a None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:13:59.564583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-00905ade-fadc-49f9-9ea6-0a1b8937ab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:13:59.680786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 714/1413] 10.222.0.71 () {58 vars in 1015 bytes} [Wed Sep 21 12:13:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:02.297497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254c5ff-184d-4968-98bc-464a8adcd01e None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:02.298086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254c5ff-184d-4968-98bc-464a8adcd01e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:02.298086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254c5ff-184d-4968-98bc-464a8adcd01e None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:02.299390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:02.304968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5832a-ff12-4088-9ee1-4409fa1fcedd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:02.305516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5832a-ff12-4088-9ee1-4409fa1fcedd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:02.305516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5832a-ff12-4088-9ee1-4409fa1fcedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:02.372460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-38e5832a-ff12-4088-9ee1-4409fa1fcedd None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:02.373672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-38e5832a-ff12-4088-9ee1-4409fa1fcedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:02.428776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:02.472361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:02.773595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254c5ff-184d-4968-98bc-464a8adcd01e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:02.775237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f254c5ff-184d-4968-98bc-464a8adcd01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:02.924094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254c5ff-184d-4968-98bc-464a8adcd01e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:02.998282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:03.038094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178258c-8703-4d22-bbc0-812eca604e87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:03.038675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178258c-8703-4d22-bbc0-812eca604e87 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:03.038957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178258c-8703-4d22-bbc0-812eca604e87 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:03.039226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178258c-8703-4d22-bbc0-812eca604e87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:03.041348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8178258c-8703-4d22-bbc0-812eca604e87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:03.130112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178258c-8703-4d22-bbc0-812eca604e87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:03.131737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178258c-8703-4d22-bbc0-812eca604e87 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:03.149266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 701/1416] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:04.289417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94555bdc-1877-4632-8a5f-5397641d3ac7 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:04.290580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94555bdc-1877-4632-8a5f-5397641d3ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:04.290580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94555bdc-1877-4632-8a5f-5397641d3ac7 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:04.291888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:14:04.298272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a889234-8f6c-4118-9a44-1d923a164ae6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:04.298699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a889234-8f6c-4118-9a44-1d923a164ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:04.298837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a889234-8f6c-4118-9a44-1d923a164ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:04.373484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6a889234-8f6c-4118-9a44-1d923a164ae6 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:04.374697 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6a889234-8f6c-4118-9a44-1d923a164ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:04.401190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:04.471273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:04.620705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94555bdc-1877-4632-8a5f-5397641d3ac7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:04.622258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94555bdc-1877-4632-8a5f-5397641d3ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:04.772622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94555bdc-1877-4632-8a5f-5397641d3ac7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:04.813287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:04.887520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478dbd7-a1a7-4fdb-b588-1d0360f76556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:04.888062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478dbd7-a1a7-4fdb-b588-1d0360f76556 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:04.888340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478dbd7-a1a7-4fdb-b588-1d0360f76556 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:04.888610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478dbd7-a1a7-4fdb-b588-1d0360f76556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:04.890219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3478dbd7-a1a7-4fdb-b588-1d0360f76556 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:04.981175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478dbd7-a1a7-4fdb-b588-1d0360f76556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:04.982763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478dbd7-a1a7-4fdb-b588-1d0360f76556 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:05.001813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 717/1419] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:06.141571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76181c1f-aad1-4245-b14b-2a77b56353e3 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:06.142130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76181c1f-aad1-4245-b14b-2a77b56353e3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:06.142130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76181c1f-aad1-4245-b14b-2a77b56353e3 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:06.143617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:06.151389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f76e4e-76fd-4058-b93f-1feb87713158 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:06.151932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f76e4e-76fd-4058-b93f-1feb87713158 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:06.152362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f76e4e-76fd-4058-b93f-1feb87713158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:06.209677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:06.232297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-97f76e4e-76fd-4058-b93f-1feb87713158 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:06.233528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97f76e4e-76fd-4058-b93f-1feb87713158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:06.330439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:06.484166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76181c1f-aad1-4245-b14b-2a77b56353e3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:06.485601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-76181c1f-aad1-4245-b14b-2a77b56353e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:06.629128 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76181c1f-aad1-4245-b14b-2a77b56353e3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:06.641080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:06.749221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ffa44-ac90-49cb-93f4-ac30314861c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:06.749753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ffa44-ac90-49cb-93f4-ac30314861c1 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:06.750035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ffa44-ac90-49cb-93f4-ac30314861c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:06.750311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ffa44-ac90-49cb-93f4-ac30314861c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:06.751951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-117ffa44-ac90-49cb-93f4-ac30314861c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:06.842792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ffa44-ac90-49cb-93f4-ac30314861c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:06.844436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ffa44-ac90-49cb-93f4-ac30314861c1 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:06.862166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 704/1422] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:07.968374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f2a0-d70d-42ee-8674-54e0e6f25767 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:07.968913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f2a0-d70d-42ee-8674-54e0e6f25767 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:07.968913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6f2a0-d70d-42ee-8674-54e0e6f25767 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:07.970250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:07.976455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a26c2-4381-48f8-8fe7-c2bcbc2505fa None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:07.976931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a26c2-4381-48f8-8fe7-c2bcbc2505fa None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:07.977035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a26c2-4381-48f8-8fe7-c2bcbc2505fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:07.997297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:08.053578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-819a26c2-4381-48f8-8fe7-c2bcbc2505fa None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:08.054700 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-819a26c2-4381-48f8-8fe7-c2bcbc2505fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:08.148934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:08.299416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6f2a0-d70d-42ee-8674-54e0e6f25767 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:08.300958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7fc6f2a0-d70d-42ee-8674-54e0e6f25767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:08.410539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:08.447458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6f2a0-d70d-42ee-8674-54e0e6f25767 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:08.560858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bb408-391f-4eac-8a11-ab5018853193 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:08.561477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bb408-391f-4eac-8a11-ab5018853193 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:08.561923 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bb408-391f-4eac-8a11-ab5018853193 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:08.562175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bb408-391f-4eac-8a11-ab5018853193 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:08.563977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0d7bb408-391f-4eac-8a11-ab5018853193 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:08.644455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:08.658263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bb408-391f-4eac-8a11-ab5018853193 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:08.659888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bb408-391f-4eac-8a11-ab5018853193 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:08.678852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 720/1425] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:09.791069 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a75cb-2625-4c9b-bf4d-fc775bfc276c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:09.791591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a75cb-2625-4c9b-bf4d-fc775bfc276c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:09.791703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a75cb-2625-4c9b-bf4d-fc775bfc276c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:09.792993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:09.799976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c5ab8-c2dc-43f9-bed3-c722a7893fe5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:09.800603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c5ab8-c2dc-43f9-bed3-c722a7893fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:09.800603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c5ab8-c2dc-43f9-bed3-c722a7893fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:09.882681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-551c5ab8-c2dc-43f9-bed3-c722a7893fe5 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:09.884506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-551c5ab8-c2dc-43f9-bed3-c722a7893fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:09.982109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:10.134908 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a75cb-2625-4c9b-bf4d-fc775bfc276c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:10.136455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-018a75cb-2625-4c9b-bf4d-fc775bfc276c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:10.223319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:10.284610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a75cb-2625-4c9b-bf4d-fc775bfc276c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:10.398896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296f4c5-b510-4146-98d9-1c95422406e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:10.399507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296f4c5-b510-4146-98d9-1c95422406e3 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:10.399821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296f4c5-b510-4146-98d9-1c95422406e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:10.400096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296f4c5-b510-4146-98d9-1c95422406e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:10.401698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a296f4c5-b510-4146-98d9-1c95422406e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:10.451859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:10.493533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296f4c5-b510-4146-98d9-1c95422406e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:10.495128 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296f4c5-b510-4146-98d9-1c95422406e3 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:10.512761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 707/1428] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:14:11.569878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84de14-7444-4862-bd78-add42e14bf06 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:11.570446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84de14-7444-4862-bd78-add42e14bf06 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:11.570446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84de14-7444-4862-bd78-add42e14bf06 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:11.571826 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:11.578178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd36f4d-685a-42cc-9e3f-87281f32ed96 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:11.578601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd36f4d-685a-42cc-9e3f-87281f32ed96 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:11.578724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd36f4d-685a-42cc-9e3f-87281f32ed96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:11.653432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7fd36f4d-685a-42cc-9e3f-87281f32ed96 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:11.654624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7fd36f4d-685a-42cc-9e3f-87281f32ed96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:11.751284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:11.903266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84de14-7444-4862-bd78-add42e14bf06 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:11.904792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dc84de14-7444-4862-bd78-add42e14bf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:11.952618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:12.052174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84de14-7444-4862-bd78-add42e14bf06 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:12.165362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513738a-8a79-4f0b-b7f3-de2d5a5396a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:12.165863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513738a-8a79-4f0b-b7f3-de2d5a5396a2 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:12.166138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513738a-8a79-4f0b-b7f3-de2d5a5396a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:12.166408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513738a-8a79-4f0b-b7f3-de2d5a5396a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:12.168038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d513738a-8a79-4f0b-b7f3-de2d5a5396a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:12.175002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:12.259166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513738a-8a79-4f0b-b7f3-de2d5a5396a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:12.260825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513738a-8a79-4f0b-b7f3-de2d5a5396a2 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:12.279373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 723/1431] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:13.387850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97c7c2-a8c8-4caf-8268-96bfaec57298 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:13.388446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97c7c2-a8c8-4caf-8268-96bfaec57298 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:13.388446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97c7c2-a8c8-4caf-8268-96bfaec57298 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:13.389752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:13.396015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e6e79-7d96-48b9-b578-896488593407 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:13.396470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e6e79-7d96-48b9-b578-896488593407 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:13.396584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e6e79-7d96-48b9-b578-896488593407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:13.475310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fc0e6e79-7d96-48b9-b578-896488593407 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:13.476565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fc0e6e79-7d96-48b9-b578-896488593407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:13.544128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:13.573936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:13.723239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97c7c2-a8c8-4caf-8268-96bfaec57298 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:13.724690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ca97c7c2-a8c8-4caf-8268-96bfaec57298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:13.742837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:13.873662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca97c7c2-a8c8-4caf-8268-96bfaec57298 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:13.961777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:13.988371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3aee09-2b21-424e-b989-ca9e3b6b9a99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:13.988906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aee09-2b21-424e-b989-ca9e3b6b9a99 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:13.989184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aee09-2b21-424e-b989-ca9e3b6b9a99 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:13.989453 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3aee09-2b21-424e-b989-ca9e3b6b9a99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:13.991139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1f3aee09-2b21-424e-b989-ca9e3b6b9a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:14.079036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aee09-2b21-424e-b989-ca9e3b6b9a99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:14.080660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3aee09-2b21-424e-b989-ca9e3b6b9a99 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:14.098510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 710/1434] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:15.172497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314202-c1a8-450e-ae84-8cdb1134dd96 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:15.173086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314202-c1a8-450e-ae84-8cdb1134dd96 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:15.173086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314202-c1a8-450e-ae84-8cdb1134dd96 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:15.174378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:15.180695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d1f3-dce3-4b87-b461-67938182fe43 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:15.181161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d1f3-dce3-4b87-b461-67938182fe43 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:15.181282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d1f3-dce3-4b87-b461-67938182fe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:15.256792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-47a8d1f3-dce3-4b87-b461-67938182fe43 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:15.258011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-47a8d1f3-dce3-4b87-b461-67938182fe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:15.299890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:15.354802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:15.503435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314202-c1a8-450e-ae84-8cdb1134dd96 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:15.505020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ee314202-c1a8-450e-ae84-8cdb1134dd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:15.649879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314202-c1a8-450e-ae84-8cdb1134dd96 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:15.705019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:15.769772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22912b66-885f-44d3-9cc2-9e7d4d22f97f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:15.770374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22912b66-885f-44d3-9cc2-9e7d4d22f97f service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:15.770731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22912b66-885f-44d3-9cc2-9e7d4d22f97f service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:15.771002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22912b66-885f-44d3-9cc2-9e7d4d22f97f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:15.772695 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22912b66-885f-44d3-9cc2-9e7d4d22f97f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:15.861897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22912b66-885f-44d3-9cc2-9e7d4d22f97f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:15.863524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22912b66-885f-44d3-9cc2-9e7d4d22f97f service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:15.882485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 726/1437] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:16.952629 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2162-a3b9-476d-88d5-0759c1d53be7 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:16.953101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2162-a3b9-476d-88d5-0759c1d53be7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:16.953196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f2162-a3b9-476d-88d5-0759c1d53be7 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:16.954520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:16.961206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7161e4-caaf-48cb-85e2-4820c046df5d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:16.962151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7161e4-caaf-48cb-85e2-4820c046df5d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:16.962255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7161e4-caaf-48cb-85e2-4820c046df5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:17.036517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5d7161e4-caaf-48cb-85e2-4820c046df5d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:17.037633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5d7161e4-caaf-48cb-85e2-4820c046df5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:17.040227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:17.133076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:17.283682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f2162-a3b9-476d-88d5-0759c1d53be7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:17.285254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ab3f2162-a3b9-476d-88d5-0759c1d53be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:17.432631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f2162-a3b9-476d-88d5-0759c1d53be7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:17.465168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:17.550025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfa002-94e7-43a4-8bac-e30c10037df7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:17.550630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfa002-94e7-43a4-8bac-e30c10037df7 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:17.550909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfa002-94e7-43a4-8bac-e30c10037df7 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:17.551177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfa002-94e7-43a4-8bac-e30c10037df7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:17.552802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5fdfa002-94e7-43a4-8bac-e30c10037df7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:17.641379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfa002-94e7-43a4-8bac-e30c10037df7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:17.642971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfa002-94e7-43a4-8bac-e30c10037df7 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:17.660599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 713/1440] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:19.739635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098156b2-6016-4690-bd40-410f135125e5 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:19.740261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098156b2-6016-4690-bd40-410f135125e5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:19.740261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098156b2-6016-4690-bd40-410f135125e5 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:19.744338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:19.747239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed91274-3531-44ac-84a4-844a2782cc37 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:19.747656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed91274-3531-44ac-84a4-844a2782cc37 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:19.747847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed91274-3531-44ac-84a4-844a2782cc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:19.789720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:19.814823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-aed91274-3531-44ac-84a4-844a2782cc37 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:19.816068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aed91274-3531-44ac-84a4-844a2782cc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:19.914992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:20.199624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098156b2-6016-4690-bd40-410f135125e5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:20.201227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-098156b2-6016-4690-bd40-410f135125e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:20.310166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098156b2-6016-4690-bd40-410f135125e5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:20.343966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:20.387026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60c503-7ab8-446b-bd6b-251242f285e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:20.387632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c503-7ab8-446b-bd6b-251242f285e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:20.387936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c503-7ab8-446b-bd6b-251242f285e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:20.388202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60c503-7ab8-446b-bd6b-251242f285e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:20.389786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cd60c503-7ab8-446b-bd6b-251242f285e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:20.505729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c503-7ab8-446b-bd6b-251242f285e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:20.507177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60c503-7ab8-446b-bd6b-251242f285e0 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:20.526319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 729/1443] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:22.837591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060cba1-c847-41aa-9eca-dd3dfceb5a4c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:22.838180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060cba1-c847-41aa-9eca-dd3dfceb5a4c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:22.838180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e060cba1-c847-41aa-9eca-dd3dfceb5a4c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:22.839469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:22.844950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18932-28d5-477f-9f14-42e6eed590fb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:22.845394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18932-28d5-477f-9f14-42e6eed590fb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:22.845507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18932-28d5-477f-9f14-42e6eed590fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:22.912723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ede18932-28d5-477f-9f14-42e6eed590fb None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:22.913932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ede18932-28d5-477f-9f14-42e6eed590fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:22.926576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:23.011803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:23.296728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060cba1-c847-41aa-9eca-dd3dfceb5a4c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:23.298264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e060cba1-c847-41aa-9eca-dd3dfceb5a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:23.401620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e060cba1-c847-41aa-9eca-dd3dfceb5a4c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:23.416661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:23.477327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b370cd5-195a-4142-9abe-35886d64ec38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:23.477926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b370cd5-195a-4142-9abe-35886d64ec38 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:23.478198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b370cd5-195a-4142-9abe-35886d64ec38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:23.478460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b370cd5-195a-4142-9abe-35886d64ec38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:23.480071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5b370cd5-195a-4142-9abe-35886d64ec38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:23.587656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b370cd5-195a-4142-9abe-35886d64ec38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:23.589123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b370cd5-195a-4142-9abe-35886d64ec38 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:23.607502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 716/1446] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:26.155226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe682-2740-4047-8347-39907138a392 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:26.155823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe682-2740-4047-8347-39907138a392 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:26.155950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe682-2740-4047-8347-39907138a392 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:26.157344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:26.163356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bc435-6943-49d5-b21c-23a8d14044a9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:26.163880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bc435-6943-49d5-b21c-23a8d14044a9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:26.163989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bc435-6943-49d5-b21c-23a8d14044a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:26.213380 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:26.231768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f42bc435-6943-49d5-b21c-23a8d14044a9 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:26.233004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f42bc435-6943-49d5-b21c-23a8d14044a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:26.330255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:14:26.611886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affe682-2740-4047-8347-39907138a392 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:26.613414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5affe682-2740-4047-8347-39907138a392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:26.715997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affe682-2740-4047-8347-39907138a392 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:26.756444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:26.793164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe302f13-29f6-48cf-808a-14f5ba5e232c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:26.793770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe302f13-29f6-48cf-808a-14f5ba5e232c service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:26.794052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe302f13-29f6-48cf-808a-14f5ba5e232c service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:26.794328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe302f13-29f6-48cf-808a-14f5ba5e232c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:26.795961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fe302f13-29f6-48cf-808a-14f5ba5e232c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:26.907371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe302f13-29f6-48cf-808a-14f5ba5e232c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:26.908823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe302f13-29f6-48cf-808a-14f5ba5e232c service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:26.927527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 732/1449] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:29.288358 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca7c1-630f-4f6a-bbae-0e527589abef None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:29.288949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca7c1-630f-4f6a-bbae-0e527589abef None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:29.288949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bca7c1-630f-4f6a-bbae-0e527589abef None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:29.290755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:29.296076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2106-3ef7-4e3c-bb34-d3badaa46348 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:29.296536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2106-3ef7-4e3c-bb34-d3badaa46348 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:29.296651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2106-3ef7-4e3c-bb34-d3badaa46348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:29.363361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-499d2106-3ef7-4e3c-bb34-d3badaa46348 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:29.364618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-499d2106-3ef7-4e3c-bb34-d3badaa46348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:29.386049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:29.463992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:29.749389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bca7c1-630f-4f6a-bbae-0e527589abef None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:29.750997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-45bca7c1-630f-4f6a-bbae-0e527589abef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:29.854919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bca7c1-630f-4f6a-bbae-0e527589abef None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:29.875720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:29.933373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a1aa6-b7eb-486d-84e3-95100cb76439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:29.933986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a1aa6-b7eb-486d-84e3-95100cb76439 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:29.934245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a1aa6-b7eb-486d-84e3-95100cb76439 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:29.934512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a1aa6-b7eb-486d-84e3-95100cb76439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:29.936124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f68a1aa6-b7eb-486d-84e3-95100cb76439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:30.045234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a1aa6-b7eb-486d-84e3-95100cb76439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:30.046586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a1aa6-b7eb-486d-84e3-95100cb76439 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:30.064898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 719/1452] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:32.302562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61159ebb-1051-4583-acfa-9de13f832b60 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:32.303128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61159ebb-1051-4583-acfa-9de13f832b60 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:32.303128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61159ebb-1051-4583-acfa-9de13f832b60 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:32.304620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:32.312032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a684c5-a88e-4a21-a202-2e741a3e8eba None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:32.312533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a684c5-a88e-4a21-a202-2e741a3e8eba None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:32.312717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a684c5-a88e-4a21-a202-2e741a3e8eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:32.376652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:32.380129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a2a684c5-a88e-4a21-a202-2e741a3e8eba None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:32.381773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a2a684c5-a88e-4a21-a202-2e741a3e8eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:32.480631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:32.767406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61159ebb-1051-4583-acfa-9de13f832b60 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:32.769004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-61159ebb-1051-4583-acfa-9de13f832b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:32.872279 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61159ebb-1051-4583-acfa-9de13f832b60 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:32.923420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:32.948621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd424e1-5390-4411-84c2-af4097da38b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:32.949221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd424e1-5390-4411-84c2-af4097da38b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:32.949497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd424e1-5390-4411-84c2-af4097da38b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:32.949766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd424e1-5390-4411-84c2-af4097da38b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:32.951362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-acd424e1-5390-4411-84c2-af4097da38b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:33.060380 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd424e1-5390-4411-84c2-af4097da38b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:33.061732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd424e1-5390-4411-84c2-af4097da38b2 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:33.079833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 735/1455] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:35.355103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8580d-8ced-4111-a106-3e64c484b9c8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:35.355641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8580d-8ced-4111-a106-3e64c484b9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:35.355641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8580d-8ced-4111-a106-3e64c484b9c8 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:35.356998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:35.362430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989331e-38d0-492b-b8ca-c01f62f0c57e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:35.362855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989331e-38d0-492b-b8ca-c01f62f0c57e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:35.362981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989331e-38d0-492b-b8ca-c01f62f0c57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:35.428924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c989331e-38d0-492b-b8ca-c01f62f0c57e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:35.430730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c989331e-38d0-492b-b8ca-c01f62f0c57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:35.462401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:35.530080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:35.811797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8580d-8ced-4111-a106-3e64c484b9c8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:35.813340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f8a8580d-8ced-4111-a106-3e64c484b9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:35.918013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8580d-8ced-4111-a106-3e64c484b9c8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:35.946922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:35.994051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d3c6d-ae13-44da-aed3-c938b7b85574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:35.994661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d3c6d-ae13-44da-aed3-c938b7b85574 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:35.994943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d3c6d-ae13-44da-aed3-c938b7b85574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:35.995216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d3c6d-ae13-44da-aed3-c938b7b85574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:35.996853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-574d3c6d-ae13-44da-aed3-c938b7b85574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:36.105564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d3c6d-ae13-44da-aed3-c938b7b85574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:36.106986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d3c6d-ae13-44da-aed3-c938b7b85574 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:36.124735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 722/1458] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:38.367460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833bf7b-1a2d-4764-be4c-ce4387b8895f None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:38.368026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833bf7b-1a2d-4764-be4c-ce4387b8895f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:38.368123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833bf7b-1a2d-4764-be4c-ce4387b8895f None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:38.369406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:38.374865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621d044-8806-4e67-9e6e-82e9ddf49e9d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:38.375262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621d044-8806-4e67-9e6e-82e9ddf49e9d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:38.375410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621d044-8806-4e67-9e6e-82e9ddf49e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:38.440098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2621d044-8806-4e67-9e6e-82e9ddf49e9d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:38.441317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2621d044-8806-4e67-9e6e-82e9ddf49e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:38.450286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:38.540731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:38.826298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833bf7b-1a2d-4764-be4c-ce4387b8895f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:38.827866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e833bf7b-1a2d-4764-be4c-ce4387b8895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:38.932396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833bf7b-1a2d-4764-be4c-ce4387b8895f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:38.988270 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:39.005885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4705c4-2a52-437a-a5a9-d6243c00d803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:39.006509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4705c4-2a52-437a-a5a9-d6243c00d803 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:39.006786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4705c4-2a52-437a-a5a9-d6243c00d803 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:39.007059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4705c4-2a52-437a-a5a9-d6243c00d803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:39.009174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cf4705c4-2a52-437a-a5a9-d6243c00d803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:39.117034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4705c4-2a52-437a-a5a9-d6243c00d803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:39.118553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4705c4-2a52-437a-a5a9-d6243c00d803 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:39.137292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 738/1461] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:41.374590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55b1af-8ce6-42b7-8e41-3155692b2a3b None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:41.375183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55b1af-8ce6-42b7-8e41-3155692b2a3b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:41.375183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55b1af-8ce6-42b7-8e41-3155692b2a3b None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:41.376594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:41.381814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a22d9a-cd18-44a1-bb98-562b9054bc91 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:41.382223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a22d9a-cd18-44a1-bb98-562b9054bc91 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:41.382433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a22d9a-cd18-44a1-bb98-562b9054bc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:41.451690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-22a22d9a-cd18-44a1-bb98-562b9054bc91 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:41.452848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22a22d9a-cd18-44a1-bb98-562b9054bc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:41.491094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:41.549574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:41.836508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55b1af-8ce6-42b7-8e41-3155692b2a3b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:41.838051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4d55b1af-8ce6-42b7-8e41-3155692b2a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:41.941450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55b1af-8ce6-42b7-8e41-3155692b2a3b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:41.979732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:42.015196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93f9ac-4d4b-4ae0-950c-a17fee1067ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:42.015778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93f9ac-4d4b-4ae0-950c-a17fee1067ca service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:42.016057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93f9ac-4d4b-4ae0-950c-a17fee1067ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:42.016328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93f9ac-4d4b-4ae0-950c-a17fee1067ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:42.017907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3b93f9ac-4d4b-4ae0-950c-a17fee1067ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:42.126056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93f9ac-4d4b-4ae0-950c-a17fee1067ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:42.127489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93f9ac-4d4b-4ae0-950c-a17fee1067ca service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:42.146436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 725/1464] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:44.394106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf53551-925c-4d10-8f5c-a5f7310b4c11 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:44.394696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf53551-925c-4d10-8f5c-a5f7310b4c11 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:44.394696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf53551-925c-4d10-8f5c-a5f7310b4c11 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:44.396045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:44.401479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849ca68-c585-47e6-b4d9-8e0e752f1f8c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:44.401906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849ca68-c585-47e6-b4d9-8e0e752f1f8c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:44.402013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849ca68-c585-47e6-b4d9-8e0e752f1f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:44.468611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8849ca68-c585-47e6-b4d9-8e0e752f1f8c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:44.469824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8849ca68-c585-47e6-b4d9-8e0e752f1f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:44.488697 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:44.568734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:44.855238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf53551-925c-4d10-8f5c-a5f7310b4c11 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:44.856933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-abf53551-925c-4d10-8f5c-a5f7310b4c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:44.962104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf53551-925c-4d10-8f5c-a5f7310b4c11 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:45.029083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:45.035887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a3307-c1e5-4aae-a508-bb14c6d004b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:45.036490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a3307-c1e5-4aae-a508-bb14c6d004b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:45.036768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a3307-c1e5-4aae-a508-bb14c6d004b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:45.037040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a3307-c1e5-4aae-a508-bb14c6d004b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:45.038639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1d0a3307-c1e5-4aae-a508-bb14c6d004b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:45.148647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a3307-c1e5-4aae-a508-bb14c6d004b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:45.150081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a3307-c1e5-4aae-a508-bb14c6d004b6 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:45.168075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 741/1467] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:47.343477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfd1c5-2a3a-44f0-87c1-804aa99f1f75 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:47.344076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfd1c5-2a3a-44f0-87c1-804aa99f1f75 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:47.344076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfd1c5-2a3a-44f0-87c1-804aa99f1f75 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:47.345396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:14:47.350735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c745ec-7401-4277-9f5e-e9728e7cfe64 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:47.351159 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c745ec-7401-4277-9f5e-e9728e7cfe64 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:47.351329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c745ec-7401-4277-9f5e-e9728e7cfe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:47.417999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-27c745ec-7401-4277-9f5e-e9728e7cfe64 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:14:47.419227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-27c745ec-7401-4277-9f5e-e9728e7cfe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:47.470588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:47.519999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:14:47.802617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfd1c5-2a3a-44f0-87c1-804aa99f1f75 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:14:47.804721 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-54bfd1c5-2a3a-44f0-87c1-804aa99f1f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:47.910059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfd1c5-2a3a-44f0-87c1-804aa99f1f75 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:14:47.964253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:47.993620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57972b7-9934-4c00-81aa-4c4975571e63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:14:47.994182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57972b7-9934-4c00-81aa-4c4975571e63 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:14:47.994465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57972b7-9934-4c00-81aa-4c4975571e63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:14:47.994739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57972b7-9934-4c00-81aa-4c4975571e63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:14:47.996427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f57972b7-9934-4c00-81aa-4c4975571e63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:14:48.105917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57972b7-9934-4c00-81aa-4c4975571e63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:14:48.107347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57972b7-9934-4c00-81aa-4c4975571e63 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:14:48.241277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 728/1470] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:14:57.518413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=22 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:14:58.238760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=66 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:09.222204 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a1ef7-1567-45c8-958b-e4782362a27d None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:09.222729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a1ef7-1567-45c8-958b-e4782362a27d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:09.222729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a1ef7-1567-45c8-958b-e4782362a27d None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:09.224108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 743/1471] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:15:09.229406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1b58-a554-4d45-a527-69eddf73e579 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:09.229839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1b58-a554-4d45-a527-69eddf73e579 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:09.229933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca1b58-a554-4d45-a527-69eddf73e579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:09.298126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9cca1b58-a554-4d45-a527-69eddf73e579 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:15:09.299339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9cca1b58-a554-4d45-a527-69eddf73e579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:09.393581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 729/1472] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:15:09.691846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628b68d-6e4a-4afd-839e-3a358863ef10 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:09.692131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628b68d-6e4a-4afd-839e-3a358863ef10 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:09.692401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628b68d-6e4a-4afd-839e-3a358863ef10 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:09.693732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 744/1473] 10.222.0.71 () {54 vars in 962 bytes} [Wed Sep 21 12:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:15:09.699384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a1cfa-baee-4190-a4c3-a57e202dc6fa None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:09.699660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a1cfa-baee-4190-a4c3-a57e202dc6fa None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:09.699949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a1cfa-baee-4190-a4c3-a57e202dc6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:09.775511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-531a1cfa-baee-4190-a4c3-a57e202dc6fa None None] MFA Rules not processed for user `56a94ccca15c462c9e7a5e88b907f420`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:15:09.776764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-531a1cfa-baee-4190-a4c3-a57e202dc6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:09.797360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:09.853267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 730/1474] 10.222.0.71 () {58 vars in 1059 bytes} [Wed Sep 21 12:15:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:15:09.862231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628b68d-6e4a-4afd-839e-3a358863ef10 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:15:09.864334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6628b68d-6e4a-4afd-839e-3a358863ef10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:09.969970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628b68d-6e4a-4afd-839e-3a358863ef10 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:15:10.045966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2e232-d11f-42cf-b7d9-6cd9484a2a1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56a94ccca15c462c9e7a5e88b907f420', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:15:10.046569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2e232-d11f-42cf-b7d9-6cd9484a2a1c service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:10.046849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2e232-d11f-42cf-b7d9-6cd9484a2a1c service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:10.047124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2e232-d11f-42cf-b7d9-6cd9484a2a1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:10.048743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c7e2e232-d11f-42cf-b7d9-6cd9484a2a1c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:10.103999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:10.159218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2e232-d11f-42cf-b7d9-6cd9484a2a1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:15:10.160682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2e232-d11f-42cf-b7d9-6cd9484a2a1c service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:15:10.179295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 745/1475] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:15:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:15:12.434101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542f1f-54fb-4826-9b87-f84b13830b18 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:12.434705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542f1f-54fb-4826-9b87-f84b13830b18 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:12.434705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a542f1f-54fb-4826-9b87-f84b13830b18 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:12.436087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 731/1476] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:15:12.441610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ba5bf-6d26-4255-a6ef-26e96d5fd2f3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:12.442041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ba5bf-6d26-4255-a6ef-26e96d5fd2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:12.442144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ba5bf-6d26-4255-a6ef-26e96d5fd2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:12.512088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-890ba5bf-6d26-4255-a6ef-26e96d5fd2f3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:15:12.513297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-890ba5bf-6d26-4255-a6ef-26e96d5fd2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:12.610115 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 746/1477] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:15:12.885177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a542f1f-54fb-4826-9b87-f84b13830b18 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:15:12.886699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2a542f1f-54fb-4826-9b87-f84b13830b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:13.011360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a542f1f-54fb-4826-9b87-f84b13830b18 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:15:13.059565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:13.103507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899a894-4b6f-46eb-960c-c2358b66954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:15:13.104154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899a894-4b6f-46eb-960c-c2358b66954d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:13.104428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899a894-4b6f-46eb-960c-c2358b66954d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:13.104695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899a894-4b6f-46eb-960c-c2358b66954d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:13.105314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899a894-4b6f-46eb-960c-c2358b66954d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:15:13.107036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-0899a894-4b6f-46eb-960c-c2358b66954d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:15:13.107036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 21 12:15:13.107036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: Sep 21 12:15:13.108191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899a894-4b6f-46eb-960c-c2358b66954d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:15:13.127087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 732/1478] 10.222.0.71 () {56 vars in 1108 bytes} [Wed Sep 21 12:15:12 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:15:15.334220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8811-2101-483c-81cf-2d55964606a7 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:15.334813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8811-2101-483c-81cf-2d55964606a7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:15.334813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8811-2101-483c-81cf-2d55964606a7 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:15.336983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 747/1479] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:15:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:15:15.342468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c2952-68c4-4121-9234-c509b4063b3b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:15.342896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c2952-68c4-4121-9234-c509b4063b3b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:15.343027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c2952-68c4-4121-9234-c509b4063b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:15.408843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1d6c2952-68c4-4121-9234-c509b4063b3b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:15:15.410359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1d6c2952-68c4-4121-9234-c509b4063b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:15.511814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 733/1480] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:15:15.805273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8811-2101-483c-81cf-2d55964606a7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:15:15.806827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c64b8811-2101-483c-81cf-2d55964606a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:15.816629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:15.955193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8811-2101-483c-81cf-2d55964606a7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:15:16.035193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:16.067039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c870ba-3321-46aa-ab1d-a443a2260110 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:15:16.067631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c870ba-3321-46aa-ab1d-a443a2260110 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:16.067930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c870ba-3321-46aa-ab1d-a443a2260110 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:16.068201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c870ba-3321-46aa-ab1d-a443a2260110 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:16.070310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e4c870ba-3321-46aa-ab1d-a443a2260110 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:16.159085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c870ba-3321-46aa-ab1d-a443a2260110 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:15:16.160708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c870ba-3321-46aa-ab1d-a443a2260110 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:15:16.179830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 748/1481] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:15:18.557767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121638-7b24-4a72-8fae-9afa986e46ba None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:18.558381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121638-7b24-4a72-8fae-9afa986e46ba None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:18.558381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef121638-7b24-4a72-8fae-9afa986e46ba None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:18.559691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 734/1482] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:15:18.565018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b0c57-3dcd-4b42-88a9-2b712fc4bd0b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:18.565461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b0c57-3dcd-4b42-88a9-2b712fc4bd0b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:18.565578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b0c57-3dcd-4b42-88a9-2b712fc4bd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:18.632694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-293b0c57-3dcd-4b42-88a9-2b712fc4bd0b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:15:18.633827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-293b0c57-3dcd-4b42-88a9-2b712fc4bd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:18.683105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:18.734631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 749/1483] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:15:19.025253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121638-7b24-4a72-8fae-9afa986e46ba None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:15:19.026852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ef121638-7b24-4a72-8fae-9afa986e46ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:19.050386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:19.131015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef121638-7b24-4a72-8fae-9afa986e46ba None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:15:19.204653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cce25c-1928-48b1-98ab-8b33575b85f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:15:19.205342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cce25c-1928-48b1-98ab-8b33575b85f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:19.205554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cce25c-1928-48b1-98ab-8b33575b85f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:19.205824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cce25c-1928-48b1-98ab-8b33575b85f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:19.207639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-08cce25c-1928-48b1-98ab-8b33575b85f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:19.289722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:19.324884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cce25c-1928-48b1-98ab-8b33575b85f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:15:19.326301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cce25c-1928-48b1-98ab-8b33575b85f6 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:15:19.344345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 735/1484] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:15:21.558154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755b840-31cd-48a9-9fda-934af509a65a None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:21.558698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755b840-31cd-48a9-9fda-934af509a65a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:21.558698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f755b840-31cd-48a9-9fda-934af509a65a None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:21.560098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 750/1485] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:15:21.565618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c00de-76da-4980-9612-2b08186dc180 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:21.566068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c00de-76da-4980-9612-2b08186dc180 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:21.566171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c00de-76da-4980-9612-2b08186dc180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:21.634576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-033c00de-76da-4980-9612-2b08186dc180 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:15:21.635841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-033c00de-76da-4980-9612-2b08186dc180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:21.733421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 736/1486] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:15:22.020967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755b840-31cd-48a9-9fda-934af509a65a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:15:22.023089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f755b840-31cd-48a9-9fda-934af509a65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:22.123572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f755b840-31cd-48a9-9fda-934af509a65a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:15:22.195728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:22.197286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8c2dd-1cf8-45c2-82d6-27f25dd8b431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:15:22.197938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8c2dd-1cf8-45c2-82d6-27f25dd8b431 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:22.198412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8c2dd-1cf8-45c2-82d6-27f25dd8b431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:22.198729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8c2dd-1cf8-45c2-82d6-27f25dd8b431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:22.200722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-91e8c2dd-1cf8-45c2-82d6-27f25dd8b431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:22.330271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8c2dd-1cf8-45c2-82d6-27f25dd8b431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:15:22.331746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8c2dd-1cf8-45c2-82d6-27f25dd8b431 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:15:22.468403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 751/1487] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:15:24.797772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89b31a-556b-4d40-a092-310743d34388 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:24.798278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89b31a-556b-4d40-a092-310743d34388 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:24.798421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89b31a-556b-4d40-a092-310743d34388 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:24.799652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 737/1488] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:15:24.805969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff5035-92fa-4fb3-85be-a3f95356eb3a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:24.805969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff5035-92fa-4fb3-85be-a3f95356eb3a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:24.806515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff5035-92fa-4fb3-85be-a3f95356eb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:24.870905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c3ff5035-92fa-4fb3-85be-a3f95356eb3a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:15:24.872135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c3ff5035-92fa-4fb3-85be-a3f95356eb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:24.925031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:24.971234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 752/1489] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:15:25.265664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89b31a-556b-4d40-a092-310743d34388 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:15:25.267196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fa89b31a-556b-4d40-a092-310743d34388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:25.303823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:25.370501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89b31a-556b-4d40-a092-310743d34388 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:15:25.442526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db27567-44fa-4a87-8111-b1682b4d99f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:15:25.443125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db27567-44fa-4a87-8111-b1682b4d99f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:25.443403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db27567-44fa-4a87-8111-b1682b4d99f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:25.443675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db27567-44fa-4a87-8111-b1682b4d99f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:25.445842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5db27567-44fa-4a87-8111-b1682b4d99f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:25.529453 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:25.558544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db27567-44fa-4a87-8111-b1682b4d99f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:15:25.560141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db27567-44fa-4a87-8111-b1682b4d99f4 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:15:25.578435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 738/1490] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:15:27.960526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef60e0a-3b12-488a-8bff-080fd3783c0b None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:27.961356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef60e0a-3b12-488a-8bff-080fd3783c0b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:27.961356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef60e0a-3b12-488a-8bff-080fd3783c0b None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:27.962741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 753/1491] 10.222.0.71 () {54 vars in 927 bytes} [Wed Sep 21 12:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:15:27.968114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ca3ab-d13b-4d2a-9ba6-1ae03d5c2c42 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:27.968555 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ca3ab-d13b-4d2a-9ba6-1ae03d5c2c42 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:27.968664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ca3ab-d13b-4d2a-9ba6-1ae03d5c2c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:28.033740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e58ca3ab-d13b-4d2a-9ba6-1ae03d5c2c42 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:15:28.034884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e58ca3ab-d13b-4d2a-9ba6-1ae03d5c2c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:28.130707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 739/1492] 10.222.0.71 () {58 vars in 1024 bytes} [Wed Sep 21 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:15:28.420387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef60e0a-3b12-488a-8bff-080fd3783c0b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:15:28.422234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7ef60e0a-3b12-488a-8bff-080fd3783c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:28.526760 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef60e0a-3b12-488a-8bff-080fd3783c0b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:15:28.606230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82376c93-5157-4867-aca2-d5e48acda275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:15:28.606835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376c93-5157-4867-aca2-d5e48acda275 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:15:28.607113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376c93-5157-4867-aca2-d5e48acda275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:15:28.607382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376c93-5157-4867-aca2-d5e48acda275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:15:28.609012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82376c93-5157-4867-aca2-d5e48acda275 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:15:28.618301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:28.723171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82376c93-5157-4867-aca2-d5e48acda275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:15:28.724541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82376c93-5157-4867-aca2-d5e48acda275 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:15:28.744522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 754/1493] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:15:38.129185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=82 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:15:38.739824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=50 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:04.070350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b238e30-6740-4a35-bc2b-8353613cafe8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:04.070946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b238e30-6740-4a35-bc2b-8353613cafe8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:04.070946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b238e30-6740-4a35-bc2b-8353613cafe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:04.146013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0b238e30-6740-4a35-bc2b-8353613cafe8 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:17:04.147216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b238e30-6740-4a35-bc2b-8353613cafe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:04.243711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 740/1494] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:17:04.251677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2b23-d35e-4f06-b753-b78156870d7f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:04.252246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2b23-d35e-4f06-b753-b78156870d7f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:04.252246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343f2b23-d35e-4f06-b753-b78156870d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:04.327455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-343f2b23-d35e-4f06-b753-b78156870d7f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:17:04.328599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-343f2b23-d35e-4f06-b753-b78156870d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:04.437698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 755/1495] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:17:04.442375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b238e30-6740-4a35-bc2b-8353613cafe8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:04.443637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b238e30-6740-4a35-bc2b-8353613cafe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:04.455127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b238e30-6740-4a35-bc2b-8353613cafe8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:04.516844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:04.555812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:04.556422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:04.556701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:04.556974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:04.557620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:17:04.557918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:04.558929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:17:04.558929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 21 12:17:04.558929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: Sep 21 12:17:04.559839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:04.571056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 741/1496] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:17:04.579521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f2b23-d35e-4f06-b753-b78156870d7f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:04.588855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343f2b23-d35e-4f06-b753-b78156870d7f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:04.650450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:04.686075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:04.686671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:04.686952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:04.687222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:04.687869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:04.689605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:17:04.689605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 21 12:17:04.689605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: Sep 21 12:17:04.690803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:04.748537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 756/1497] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:17:04 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:17:04.752741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:04.754859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:04.789398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16106062-0ea7-4bce-86c3-3e44f34191e5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:04.901798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:04.902315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:04.902595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:04.902866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:04.903487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:04.904537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:17:04.904537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 21 12:17:04.904537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: Sep 21 12:17:04.905754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:04.949046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 742/1498] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:17:04 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:17:04.953621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:04.955140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:05.017495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c354207-0945-4330-ab6f-ae05b58e9c7e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:05.106747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:05.119740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:05.120368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:05.120639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:05.120905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:05.121571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:05.122568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:17:05.122568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 21 12:17:05.122568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: Sep 21 12:17:05.123452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:05.140181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 757/1499] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:17:05.144588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:05.146553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:05.158031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a8879-91ca-4dc7-8fab-afc31067f3f5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:05.183499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:05.253156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16f106-8f4a-4fbc-8b65-a6693709891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:05.253769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16f106-8f4a-4fbc-8b65-a6693709891e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:05.254042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16f106-8f4a-4fbc-8b65-a6693709891e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:05.254311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16f106-8f4a-4fbc-8b65-a6693709891e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:05.254941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16f106-8f4a-4fbc-8b65-a6693709891e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:05.255977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-5b16f106-8f4a-4fbc-8b65-a6693709891e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:17:05.255977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 21 12:17:05.255977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: Sep 21 12:17:05.256868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b16f106-8f4a-4fbc-8b65-a6693709891e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:05.268994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 743/1500] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:17:05.273507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:05.275058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:05.308901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424da53-7159-40db-90d0-a224340ee0db admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:05.402499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec514661-317e-4342-9f4a-b27389d5c284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:05.403156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec514661-317e-4342-9f4a-b27389d5c284 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:05.403434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec514661-317e-4342-9f4a-b27389d5c284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:05.403705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec514661-317e-4342-9f4a-b27389d5c284 admin admin] PATH_INFO: `/v3/projects/f8f3213839194672bb9aa48e24d73378/users/90868f206ea44328bb6dea3db865d084/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:05.427956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec514661-317e-4342-9f4a-b27389d5c284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f3213839194672bb9aa48e24d73378, user_id=90868f206ea44328bb6dea3db865d084, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:05.428928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-ec514661-317e-4342-9f4a-b27389d5c284 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:17:05.428928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 21 12:17:05.428928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: Sep 21 12:17:05.430879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec514661-317e-4342-9f4a-b27389d5c284 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:05.455157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 758/1501] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:17:05 2022] PUT /identity/v3/projects/f8f3213839194672bb9aa48e24d73378/users/90868f206ea44328bb6dea3db865d084/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:17:05.468495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c1c4e-43a8-4da3-a6df-d065c656ad64 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:05.468783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c1c4e-43a8-4da3-a6df-d065c656ad64 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:05.469070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c1c4e-43a8-4da3-a6df-d065c656ad64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:05.525992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-821c1c4e-43a8-4da3-a6df-d065c656ad64 None None] MFA Rules not processed for user `90868f206ea44328bb6dea3db865d084`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:17:05.527181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-821c1c4e-43a8-4da3-a6df-d065c656ad64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:05.604868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 744/1502] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:17:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:17:05.618204 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6f42d-9c27-4585-9de0-5b2e161056cd None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:05.618497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6f42d-9c27-4585-9de0-5b2e161056cd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:05.618774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6f42d-9c27-4585-9de0-5b2e161056cd None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:05.620153 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 759/1503] 10.222.0.71 () {54 vars in 963 bytes} [Wed Sep 21 12:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:17:05.625227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a288fd-9b4b-4012-8f3a-066e59ddd484 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:05.625544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a288fd-9b4b-4012-8f3a-066e59ddd484 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:05.625824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a288fd-9b4b-4012-8f3a-066e59ddd484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:05.658601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:05.702823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a1a288fd-9b4b-4012-8f3a-066e59ddd484 None None] MFA Rules not processed for user `5c84b440821446e3a7ce97b2c68f815e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:17:05.703971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a1a288fd-9b4b-4012-8f3a-066e59ddd484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:05.779333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 745/1504] 10.222.0.71 () {58 vars in 1060 bytes} [Wed Sep 21 12:17:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:17:05.787856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6f42d-9c27-4585-9de0-5b2e161056cd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:05.789406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-74a6f42d-9c27-4585-9de0-5b2e161056cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:05.839732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:05.894921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6f42d-9c27-4585-9de0-5b2e161056cd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:05.967426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:05.967963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:05.968240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:05.968505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:05.970077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:06.066404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:06.067849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:06.071242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:06.087097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 760/1505] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:17:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:17:06.132040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce5752-06f0-4ab4-9ce4-ecbb06083dd8 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:06.132324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce5752-06f0-4ab4-9ce4-ecbb06083dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:06.132595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce5752-06f0-4ab4-9ce4-ecbb06083dd8 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:06.133876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 746/1506] 10.222.0.71 () {52 vars in 830 bytes} [Wed Sep 21 12:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:17:06.137513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:06.138716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:06.248278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbe4b20-0346-4114-b3fb-3778c0b29b92 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:06.349059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:06.349680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:06.349958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:06.350228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] PATH_INFO: `/v3/users/90868f206ea44328bb6dea3db865d084` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:06.364519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90868f206ea44328bb6dea3db865d084)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:06.365559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:17:06.365559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 21 12:17:06.365559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: Sep 21 12:17:06.366777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:06.376502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:06.434391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90868f206ea44328bb6dea3db865d084'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:06.444180 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90868f206ea44328bb6dea3db865d084'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:06.444703 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90868f206ea44328bb6dea3db865d084'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:06.445128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90868f206ea44328bb6dea3db865d084'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:06.459055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90868f206ea44328bb6dea3db865d084'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:06.494142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 761/1507] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:17:06 2022] DELETE /identity/v3/users/90868f206ea44328bb6dea3db865d084 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:17:06.499680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e31844-68d6-4122-9437-c56760d67253 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:06.500208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e31844-68d6-4122-9437-c56760d67253 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:06.500208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e31844-68d6-4122-9437-c56760d67253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:06.558725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-95e31844-68d6-4122-9437-c56760d67253 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:17:06.560039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-95e31844-68d6-4122-9437-c56760d67253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:06.637020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:06.650133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 747/1508] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:17:06.661342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:06.662585 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:06.765380 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec1d2f-7b5c-48eb-b2c5-3ea68f139279 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:06.820343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:06.841208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0f530-85a9-41c0-8d8c-03292a824fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:06.841818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0f530-85a9-41c0-8d8c-03292a824fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:06.842095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0f530-85a9-41c0-8d8c-03292a824fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:06.842366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0f530-85a9-41c0-8d8c-03292a824fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:06.843998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97e0f530-85a9-41c0-8d8c-03292a824fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:06.952534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0f530-85a9-41c0-8d8c-03292a824fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:06.953973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0f530-85a9-41c0-8d8c-03292a824fd2 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:06.971806 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 762/1509] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:17:07.301682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e31844-68d6-4122-9437-c56760d67253 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:17:07.302916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-95e31844-68d6-4122-9437-c56760d67253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:17:07.314863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e31844-68d6-4122-9437-c56760d67253 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:17:07.415434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:17:07.416091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:17:07.416369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:17:07.416661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] PATH_INFO: `/v3/projects/f8f3213839194672bb9aa48e24d73378` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:17:07.426488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8f3213839194672bb9aa48e24d73378)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:17:07.427495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:17:07.427495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 21 12:17:07.427495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: Sep 21 12:17:07.428734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:17:07.488022 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8f3213839194672bb9aa48e24d73378', 'request_id': 'req-507aace4-d72f-4823-bc27-9c2321a5d552'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:07.500761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8f3213839194672bb9aa48e24d73378', 'request_id': 'req-507aace4-d72f-4823-bc27-9c2321a5d552'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:07.515565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8f3213839194672bb9aa48e24d73378', 'request_id': 'req-507aace4-d72f-4823-bc27-9c2321a5d552'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:07.541313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] The token cache is being invalidate because project f8f3213839194672bb9aa48e24d73378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:17:07.541689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:07.542005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-507aace4-d72f-4823-bc27-9c2321a5d552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:17:07.543459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 748/1510] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:17:07 2022] DELETE /identity/v3/projects/f8f3213839194672bb9aa48e24d73378 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:17:16.969545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=62 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:17:17.538730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:18:38.894433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ab170-ed47-495d-975e-ce20ec7cb126 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:38.894986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ab170-ed47-495d-975e-ce20ec7cb126 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:38.894986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ab170-ed47-495d-975e-ce20ec7cb126 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:38.896296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 12:18:38.901246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163eb46-87d5-4651-8562-c075467ff17c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:38.901683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163eb46-87d5-4651-8562-c075467ff17c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:38.901780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163eb46-87d5-4651-8562-c075467ff17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:38.982635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8163eb46-87d5-4651-8562-c075467ff17c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:18:38.983887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8163eb46-87d5-4651-8562-c075467ff17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:18:39.086695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:18:39.242269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ab170-ed47-495d-975e-ce20ec7cb126 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:18:39.244378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d74ab170-ed47-495d-975e-ce20ec7cb126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:18:39.361634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ab170-ed47-495d-975e-ce20ec7cb126 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:18:39.367008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa49e00-fd6d-4efb-809f-acbd006f1dfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:18:39.367607 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa49e00-fd6d-4efb-809f-acbd006f1dfa service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:39.367916 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa49e00-fd6d-4efb-809f-acbd006f1dfa service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:39.368189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa49e00-fd6d-4efb-809f-acbd006f1dfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:39.376372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa49e00-fd6d-4efb-809f-acbd006f1dfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:18:39.377992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa49e00-fd6d-4efb-809f-acbd006f1dfa service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:18:39.387289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 764/1513] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:18:40.477341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1d665-ba09-4f91-a21f-97575466e179 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:40.477926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1d665-ba09-4f91-a21f-97575466e179 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:40.477926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1d665-ba09-4f91-a21f-97575466e179 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:40.479247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:18:40.485651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06dab7-0d0f-464e-953a-6b61c8172fa6 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:40.486815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06dab7-0d0f-464e-953a-6b61c8172fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:40.486815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06dab7-0d0f-464e-953a-6b61c8172fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:40.509842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7e06dab7-0d0f-464e-953a-6b61c8172fa6 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:18:40.511050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7e06dab7-0d0f-464e-953a-6b61c8172fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:18:40.519005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:18:40.669786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1d665-ba09-4f91-a21f-97575466e179 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:18:40.684612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1d665-ba09-4f91-a21f-97575466e179 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:18:40.690134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc045be-c745-4fba-a42f-d2d41f7d3eb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:18:40.690719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc045be-c745-4fba-a42f-d2d41f7d3eb3 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:40.690994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc045be-c745-4fba-a42f-d2d41f7d3eb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:40.691260 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc045be-c745-4fba-a42f-d2d41f7d3eb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:40.699078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc045be-c745-4fba-a42f-d2d41f7d3eb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:18:40.700705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc045be-c745-4fba-a42f-d2d41f7d3eb3 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:18:40.709713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 751/1516] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:18:41.643170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb207bb8-8850-4f91-8c5c-20d20df3f777 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:41.643742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb207bb8-8850-4f91-8c5c-20d20df3f777 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:41.643742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb207bb8-8850-4f91-8c5c-20d20df3f777 None None] PATH_INFO: `/v3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:41.645037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 766/1517] 10.222.0.66 () {54 vars in 925 bytes} [Wed Sep 21 12:18:41 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:18:41.651478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149529e4-e917-4ac9-9a7d-b7e9e985cbec None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:41.651953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149529e4-e917-4ac9-9a7d-b7e9e985cbec None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:41.652062 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149529e4-e917-4ac9-9a7d-b7e9e985cbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:41.712515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-149529e4-e917-4ac9-9a7d-b7e9e985cbec None None] MFA Rules not processed for user `eac29fd43f1f4bbd987950b90385200b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:18:41.713624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-149529e4-e917-4ac9-9a7d-b7e9e985cbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:18:41.730549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:18:41.787055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 752/1518] 10.222.0.66 () {58 vars in 1015 bytes} [Wed Sep 21 12:18:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:18:50.503818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=56 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:18:51.784205 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=23 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:18:51.846750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a322e-b9d8-43c8-824f-dc46bef30174 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:51.847281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a322e-b9d8-43c8-824f-dc46bef30174 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:51.847281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a322e-b9d8-43c8-824f-dc46bef30174 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:51.848645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 767/1519] 10.222.0.71 () {54 vars in 961 bytes} [Wed Sep 21 12:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:18:51.853804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da08109-0395-431d-8bb9-e60ff8786473 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:51.854093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da08109-0395-431d-8bb9-e60ff8786473 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:51.854386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da08109-0395-431d-8bb9-e60ff8786473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:51.874765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4da08109-0395-431d-8bb9-e60ff8786473 None None] MFA Rules not processed for user `eac29fd43f1f4bbd987950b90385200b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:18:51.876477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4da08109-0395-431d-8bb9-e60ff8786473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:18:51.886209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 753/1520] 10.222.0.71 () {58 vars in 1058 bytes} [Wed Sep 21 12:18:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:18:51.894344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a322e-b9d8-43c8-824f-dc46bef30174 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:18:51.908454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a322e-b9d8-43c8-824f-dc46bef30174 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:18:51.913599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f358cf4-b9b0-4656-a6ee-65552487a1eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:18:51.914218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f358cf4-b9b0-4656-a6ee-65552487a1eb service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:51.914499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f358cf4-b9b0-4656-a6ee-65552487a1eb service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:51.914775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f358cf4-b9b0-4656-a6ee-65552487a1eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:51.923312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f358cf4-b9b0-4656-a6ee-65552487a1eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:18:51.924925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f358cf4-b9b0-4656-a6ee-65552487a1eb service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:18:51.934823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 768/1521] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:18:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:18:52.206070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457d76c-fb15-4ec3-a04d-55f744fa61a5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:52.206281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457d76c-fb15-4ec3-a04d-55f744fa61a5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:52.206528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457d76c-fb15-4ec3-a04d-55f744fa61a5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:52.207922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:18:52.214376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44b2d3-1044-483c-bf74-a7b1126c2d7e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:52.214655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44b2d3-1044-483c-bf74-a7b1126c2d7e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:52.214926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44b2d3-1044-483c-bf74-a7b1126c2d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:52.233768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4b44b2d3-1044-483c-bf74-a7b1126c2d7e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:18:52.234967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b44b2d3-1044-483c-bf74-a7b1126c2d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:18:52.243308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:18:52.396508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457d76c-fb15-4ec3-a04d-55f744fa61a5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:18:52.412660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457d76c-fb15-4ec3-a04d-55f744fa61a5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:18:52.418082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b09213-dcb3-4df5-a66b-4e85e4fb785c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:18:52.418677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b09213-dcb3-4df5-a66b-4e85e4fb785c service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:52.418956 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b09213-dcb3-4df5-a66b-4e85e4fb785c service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:52.419226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b09213-dcb3-4df5-a66b-4e85e4fb785c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:52.428788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b09213-dcb3-4df5-a66b-4e85e4fb785c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:18:52.430420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b09213-dcb3-4df5-a66b-4e85e4fb785c service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:18:52.440288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 755/1524] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:18:53.548484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a1b78-7029-42b3-a56b-fa5019702a1c None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:53.549096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a1b78-7029-42b3-a56b-fa5019702a1c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:53.549096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a1b78-7029-42b3-a56b-fa5019702a1c None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:53.550314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 12:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:18:53.556720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5e68d-d7fa-4334-bf0a-96d3c099fb37 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:53.557159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5e68d-d7fa-4334-bf0a-96d3c099fb37 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:53.557259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5e68d-d7fa-4334-bf0a-96d3c099fb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:53.577911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cfe5e68d-d7fa-4334-bf0a-96d3c099fb37 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:18:53.579848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cfe5e68d-d7fa-4334-bf0a-96d3c099fb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:18:53.587869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 12:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:18:53.738952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a1b78-7029-42b3-a56b-fa5019702a1c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:18:53.753155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a1b78-7029-42b3-a56b-fa5019702a1c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:18:53.758701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1abc4-dedc-46bf-8c0a-7dd69c3a8311 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:18:53.759296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1abc4-dedc-46bf-8c0a-7dd69c3a8311 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:53.759577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1abc4-dedc-46bf-8c0a-7dd69c3a8311 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:53.759872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1abc4-dedc-46bf-8c0a-7dd69c3a8311 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:53.768797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1abc4-dedc-46bf-8c0a-7dd69c3a8311 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:18:53.770386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1abc4-dedc-46bf-8c0a-7dd69c3a8311 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:18:53.779696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 771/1527] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:18:57.833383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d4680-6b8a-4c60-9109-bae2577ce90c None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:57.833945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d4680-6b8a-4c60-9109-bae2577ce90c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:57.833945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d4680-6b8a-4c60-9109-bae2577ce90c None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:57.835496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:18:57.840695 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b5619-30b7-494f-9a57-9e663c8eab08 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:57.841127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b5619-30b7-494f-9a57-9e663c8eab08 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:57.841236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b5619-30b7-494f-9a57-9e663c8eab08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:57.860595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-296b5619-30b7-494f-9a57-9e663c8eab08 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:18:57.861799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-296b5619-30b7-494f-9a57-9e663c8eab08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:18:57.869609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:18:58.154874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d4680-6b8a-4c60-9109-bae2577ce90c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:18:58.156608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f64d4680-6b8a-4c60-9109-bae2577ce90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:18:58.234259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d4680-6b8a-4c60-9109-bae2577ce90c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:18:58.238403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4a849-315e-4bc1-a10c-50b055ba427d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:18:58.239000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4a849-315e-4bc1-a10c-50b055ba427d service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:18:58.239280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4a849-315e-4bc1-a10c-50b055ba427d service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:18:58.239553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4a849-315e-4bc1-a10c-50b055ba427d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:18:58.248381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4a849-315e-4bc1-a10c-50b055ba427d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:18:58.249793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4a849-315e-4bc1-a10c-50b055ba427d service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:18:58.259644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 758/1530] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:19:00.467040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b825353-7a61-419b-bf26-d6aeb35edefe None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:19:00.467674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b825353-7a61-419b-bf26-d6aeb35edefe None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:19:00.467674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b825353-7a61-419b-bf26-d6aeb35edefe None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:19:00.468996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:19:00.474214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9565cf-4465-4c71-90cd-94fbfa8ff8f0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:19:00.474647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9565cf-4465-4c71-90cd-94fbfa8ff8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:19:00.474763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9565cf-4465-4c71-90cd-94fbfa8ff8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:19:00.495191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1f9565cf-4465-4c71-90cd-94fbfa8ff8f0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:19:00.496445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1f9565cf-4465-4c71-90cd-94fbfa8ff8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:19:00.505069 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:19:00.791296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b825353-7a61-419b-bf26-d6aeb35edefe None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:19:00.792981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b825353-7a61-419b-bf26-d6aeb35edefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:19:00.806201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b825353-7a61-419b-bf26-d6aeb35edefe None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:19:00.809668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd2173-b14f-4258-80b2-441423873fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:19:00.810257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd2173-b14f-4258-80b2-441423873fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:19:00.810532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd2173-b14f-4258-80b2-441423873fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:19:00.810802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd2173-b14f-4258-80b2-441423873fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:19:00.819791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd2173-b14f-4258-80b2-441423873fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:19:00.821232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd2173-b14f-4258-80b2-441423873fc8 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:19:00.830573 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 774/1533] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:19:04.630830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f5cfd-5da6-4659-970c-d6753b535dcf None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:19:04.631409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f5cfd-5da6-4659-970c-d6753b535dcf None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:19:04.631409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f5cfd-5da6-4659-970c-d6753b535dcf None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:19:04.632767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:19:04.638169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138b1cf-5ba1-4f1a-8f69-b6261a021d63 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:19:04.638576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138b1cf-5ba1-4f1a-8f69-b6261a021d63 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:19:04.638719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138b1cf-5ba1-4f1a-8f69-b6261a021d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:19:04.658609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f138b1cf-5ba1-4f1a-8f69-b6261a021d63 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:19:04.659857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f138b1cf-5ba1-4f1a-8f69-b6261a021d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:19:04.668504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:19:04.973126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f5cfd-5da6-4659-970c-d6753b535dcf None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:19:04.974807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-789f5cfd-5da6-4659-970c-d6753b535dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:19:05.052796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f5cfd-5da6-4659-970c-d6753b535dcf None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:19:05.056428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964e2f43-d4fd-4422-8c32-99b9a9c0e78c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:19:05.057027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e2f43-d4fd-4422-8c32-99b9a9c0e78c service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:19:05.057304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e2f43-d4fd-4422-8c32-99b9a9c0e78c service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:19:05.057575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964e2f43-d4fd-4422-8c32-99b9a9c0e78c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:19:05.066329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e2f43-d4fd-4422-8c32-99b9a9c0e78c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:19:05.067736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964e2f43-d4fd-4422-8c32-99b9a9c0e78c service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:19:05.077526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 761/1536] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:19:07.251567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e081f5-352b-40b2-9e6b-c77caec49e94 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:19:07.252198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e081f5-352b-40b2-9e6b-c77caec49e94 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:19:07.252198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e081f5-352b-40b2-9e6b-c77caec49e94 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:19:07.253576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 12:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:19:07.258854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f203544-5291-4e4f-a191-ce1e39cdf96e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:19:07.259261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f203544-5291-4e4f-a191-ce1e39cdf96e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:19:07.259375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f203544-5291-4e4f-a191-ce1e39cdf96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:19:07.279068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3f203544-5291-4e4f-a191-ce1e39cdf96e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:19:07.280283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3f203544-5291-4e4f-a191-ce1e39cdf96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:19:07.288515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 12:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:19:07.588704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e081f5-352b-40b2-9e6b-c77caec49e94 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:19:07.590372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-94e081f5-352b-40b2-9e6b-c77caec49e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:19:07.603559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e081f5-352b-40b2-9e6b-c77caec49e94 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:19:07.607094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618c261-ff0d-4c34-89e2-712f1dbcd264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:19:07.607719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618c261-ff0d-4c34-89e2-712f1dbcd264 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:19:07.608038 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618c261-ff0d-4c34-89e2-712f1dbcd264 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:19:07.608336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618c261-ff0d-4c34-89e2-712f1dbcd264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:19:07.617630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618c261-ff0d-4c34-89e2-712f1dbcd264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:19:07.619088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618c261-ff0d-4c34-89e2-712f1dbcd264 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:19:07.629668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 777/1539] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:19:17.274145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:19:17.624538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=33 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:34.070891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e12ac-49db-40ae-8ecd-95a7c09f8b4b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.071986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e12ac-49db-40ae-8ecd-95a7c09f8b4b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.071986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e12ac-49db-40ae-8ecd-95a7c09f8b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.096705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7f0e12ac-49db-40ae-8ecd-95a7c09f8b4b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.097907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7f0e12ac-49db-40ae-8ecd-95a7c09f8b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.111013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 763/1540] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.119484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297d331-4623-4eae-9cc0-e17c62613150 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.120469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297d331-4623-4eae-9cc0-e17c62613150 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.120989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297d331-4623-4eae-9cc0-e17c62613150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.152911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1297d331-4623-4eae-9cc0-e17c62613150 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.154537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1297d331-4623-4eae-9cc0-e17c62613150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.170008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 778/1541] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.175387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e12ac-49db-40ae-8ecd-95a7c09f8b4b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.190241 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e12ac-49db-40ae-8ecd-95a7c09f8b4b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:34.200173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a086e6-56cf-408e-9fe7-3c5b00c08c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:34.201204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a086e6-56cf-408e-9fe7-3c5b00c08c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.201684 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a086e6-56cf-408e-9fe7-3c5b00c08c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.202159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a086e6-56cf-408e-9fe7-3c5b00c08c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.203086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a086e6-56cf-408e-9fe7-3c5b00c08c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:34.203612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a086e6-56cf-408e-9fe7-3c5b00c08c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:34.205624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a086e6-56cf-408e-9fe7-3c5b00c08c39 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:34.221103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 764/1542] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:34.229221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297d331-4623-4eae-9cc0-e17c62613150 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.247153 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297d331-4623-4eae-9cc0-e17c62613150 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:34.257520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21aac5-8cf8-4f3b-bd42-de66c6ebf9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:34.258153 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21aac5-8cf8-4f3b-bd42-de66c6ebf9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.258434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21aac5-8cf8-4f3b-bd42-de66c6ebf9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.258706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21aac5-8cf8-4f3b-bd42-de66c6ebf9c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.259383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21aac5-8cf8-4f3b-bd42-de66c6ebf9c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:34.261393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21aac5-8cf8-4f3b-bd42-de66c6ebf9c2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:34.338520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 779/1543] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:34.342718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a086e6-56cf-408e-9fe7-3c5b00c08c39 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.358502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a086e6-56cf-408e-9fe7-3c5b00c08c39 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:34.427662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143e73-4c23-4f6f-ba09-07361c86870d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.427992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143e73-4c23-4f6f-ba09-07361c86870d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.428252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143e73-4c23-4f6f-ba09-07361c86870d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.435936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fdca5-3446-4bd4-afa3-957bd321a64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:34.436981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fdca5-3446-4bd4-afa3-957bd321a64b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.437462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fdca5-3446-4bd4-afa3-957bd321a64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.437931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fdca5-3446-4bd4-afa3-957bd321a64b admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.438787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60fdca5-3446-4bd4-afa3-957bd321a64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:34.441052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60fdca5-3446-4bd4-afa3-957bd321a64b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:34.461392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-13143e73-4c23-4f6f-ba09-07361c86870d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.462682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-13143e73-4c23-4f6f-ba09-07361c86870d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.478173 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 780/1544] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.481104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 765/1545] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:34.485536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143e73-4c23-4f6f-ba09-07361c86870d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.488091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba9b2d-4027-41d0-b1fd-2354108dcb4a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.488743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba9b2d-4027-41d0-b1fd-2354108dcb4a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.489469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba9b2d-4027-41d0-b1fd-2354108dcb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.502578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143e73-4c23-4f6f-ba09-07361c86870d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:34.516539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b240aae-54a0-4fed-b2d6-02cb005b3d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:34.517295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b240aae-54a0-4fed-b2d6-02cb005b3d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.517583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b240aae-54a0-4fed-b2d6-02cb005b3d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.517858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b240aae-54a0-4fed-b2d6-02cb005b3d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.518566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b240aae-54a0-4fed-b2d6-02cb005b3d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:34.520302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b240aae-54a0-4fed-b2d6-02cb005b3d08 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:34.522937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0cba9b2d-4027-41d0-b1fd-2354108dcb4a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.526846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0cba9b2d-4027-41d0-b1fd-2354108dcb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.536479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 781/1546] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:34.539164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 766/1547] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:20:34.540477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b240aae-54a0-4fed-b2d6-02cb005b3d08 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.543412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba9b2d-4027-41d0-b1fd-2354108dcb4a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.555251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b240aae-54a0-4fed-b2d6-02cb005b3d08 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:34.557715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba9b2d-4027-41d0-b1fd-2354108dcb4a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:34.568174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355197d-40cb-4610-8ac5-dfb5f7e6b651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:34.569163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355197d-40cb-4610-8ac5-dfb5f7e6b651 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.569667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355197d-40cb-4610-8ac5-dfb5f7e6b651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.570163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355197d-40cb-4610-8ac5-dfb5f7e6b651 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.570898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae76228-3626-4796-be6c-7b470d295e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:34.571301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355197d-40cb-4610-8ac5-dfb5f7e6b651 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:34.571527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae76228-3626-4796-be6c-7b470d295e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.571843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae76228-3626-4796-be6c-7b470d295e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.572260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae76228-3626-4796-be6c-7b470d295e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.572472 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355197d-40cb-4610-8ac5-dfb5f7e6b651 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:34.572911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae76228-3626-4796-be6c-7b470d295e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:34.574521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae76228-3626-4796-be6c-7b470d295e8c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:34.574744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355197d-40cb-4610-8ac5-dfb5f7e6b651 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:34.593648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 782/1548] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:34.594323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 767/1549] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:20:34.597777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae76228-3626-4796-be6c-7b470d295e8c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.601942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355197d-40cb-4610-8ac5-dfb5f7e6b651 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.616050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae76228-3626-4796-be6c-7b470d295e8c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:34.617735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355197d-40cb-4610-8ac5-dfb5f7e6b651 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:34.626725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c121c6b-06d2-4d31-a449-ff87ecd5def2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:34.627807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121c6b-06d2-4d31-a449-ff87ecd5def2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.628312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121c6b-06d2-4d31-a449-ff87ecd5def2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.628441 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee5c04-7cf6-48bf-aef4-214f92490910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:34.628956 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c121c6b-06d2-4d31-a449-ff87ecd5def2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.629144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5c04-7cf6-48bf-aef4-214f92490910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.629422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5c04-7cf6-48bf-aef4-214f92490910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.629705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5c04-7cf6-48bf-aef4-214f92490910 admin admin] PATH_INFO: `/v3/projects/472d4c74ede24fd691d42b3185e919b2/users/c2211bc31f8f49f39623ee643e6d812d/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.630492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c121c6b-06d2-4d31-a449-ff87ecd5def2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:34.631722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-4c121c6b-06d2-4d31-a449-ff87ecd5def2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:20:34.631722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 21 12:20:34.631722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: Sep 21 12:20:34.635047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c121c6b-06d2-4d31-a449-ff87ecd5def2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:34.676686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee5c04-7cf6-48bf-aef4-214f92490910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=472d4c74ede24fd691d42b3185e919b2, user_id=c2211bc31f8f49f39623ee643e6d812d, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:34.679515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee5c04-7cf6-48bf-aef4-214f92490910 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:34.698362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 768/1550] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:20:34.704098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffb9eb-2d3b-4978-809c-43bf683ee032 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.704733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffb9eb-2d3b-4978-809c-43bf683ee032 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.705229 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ffb9eb-2d3b-4978-809c-43bf683ee032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.719254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 783/1551] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:34 2022] PUT /identity/v3/projects/472d4c74ede24fd691d42b3185e919b2/users/c2211bc31f8f49f39623ee643e6d812d/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:34.720141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee5c04-7cf6-48bf-aef4-214f92490910 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.736502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a0ffb9eb-2d3b-4978-809c-43bf683ee032 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.738101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a0ffb9eb-2d3b-4978-809c-43bf683ee032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.806486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee5c04-7cf6-48bf-aef4-214f92490910 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:34.808100 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 769/1552] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.809574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3269ce-6132-4405-b3d8-79bfb224cd48 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.809864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3269ce-6132-4405-b3d8-79bfb224cd48 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.810136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3269ce-6132-4405-b3d8-79bfb224cd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.817073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a7658-39b9-40b6-82c1-b197d8fa001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:34.817696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a7658-39b9-40b6-82c1-b197d8fa001f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.817976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a7658-39b9-40b6-82c1-b197d8fa001f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.818250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a7658-39b9-40b6-82c1-b197d8fa001f admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.818883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a7658-39b9-40b6-82c1-b197d8fa001f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:34.820561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-025a7658-39b9-40b6-82c1-b197d8fa001f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:20:34.820561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 21 12:20:34.820561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: Sep 21 12:20:34.821171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a7658-39b9-40b6-82c1-b197d8fa001f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:34.830015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3e3269ce-6132-4405-b3d8-79bfb224cd48 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.831219 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3e3269ce-6132-4405-b3d8-79bfb224cd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.839717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 770/1553] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.840921 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e844f7-b53e-43c7-acad-6c8606b3f852 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.841208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e844f7-b53e-43c7-acad-6c8606b3f852 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.841502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e844f7-b53e-43c7-acad-6c8606b3f852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.849021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 784/1554] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:34.849433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d495-1e16-4a37-a1d0-329541ec8bfd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.849876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d495-1e16-4a37-a1d0-329541ec8bfd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.850188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d495-1e16-4a37-a1d0-329541ec8bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.869709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e414d495-1e16-4a37-a1d0-329541ec8bfd None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.870936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e414d495-1e16-4a37-a1d0-329541ec8bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.872043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-61e844f7-b53e-43c7-acad-6c8606b3f852 None None] MFA Rules not processed for user `c2211bc31f8f49f39623ee643e6d812d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.873246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-61e844f7-b53e-43c7-acad-6c8606b3f852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.879501 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 785/1555] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.880905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370f50-5a05-4f84-815d-41f3a80732dc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.881342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370f50-5a05-4f84-815d-41f3a80732dc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.881571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370f50-5a05-4f84-815d-41f3a80732dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.886094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:34.908540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e6370f50-5a05-4f84-815d-41f3a80732dc None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.910216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e6370f50-5a05-4f84-815d-41f3a80732dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.918816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 786/1556] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.919832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe407fca-f93f-4d8e-a689-fe1d13b06cca None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.920119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe407fca-f93f-4d8e-a689-fe1d13b06cca None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.920694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe407fca-f93f-4d8e-a689-fe1d13b06cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.939640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fe407fca-f93f-4d8e-a689-fe1d13b06cca None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.940915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fe407fca-f93f-4d8e-a689-fe1d13b06cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.946040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 771/1557] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.947610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe78610-0157-4838-acc4-3446578b1581 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.947937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe78610-0157-4838-acc4-3446578b1581 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.948230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe78610-0157-4838-acc4-3446578b1581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.949208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 787/1558] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.953794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05cdbe-211e-4842-b4ff-80165317acd7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.954085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05cdbe-211e-4842-b4ff-80165317acd7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.954368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05cdbe-211e-4842-b4ff-80165317acd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.970074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3fe78610-0157-4838-acc4-3446578b1581 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.971297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3fe78610-0157-4838-acc4-3446578b1581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.974437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5e05cdbe-211e-4842-b4ff-80165317acd7 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:34.975604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e05cdbe-211e-4842-b4ff-80165317acd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:34.979152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 772/1559] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.982574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17353f5-0c7c-4416-b571-5bb8f185a6c0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:34.982574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17353f5-0c7c-4416-b571-5bb8f185a6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:34.982574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17353f5-0c7c-4416-b571-5bb8f185a6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:34.986857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 788/1560] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:34.987673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05cdbe-211e-4842-b4ff-80165317acd7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:34.997677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05cdbe-211e-4842-b4ff-80165317acd7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.002764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876d2d1-24b8-492e-9272-b4321c995f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.002917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b17353f5-0c7c-4416-b571-5bb8f185a6c0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:35.003358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876d2d1-24b8-492e-9272-b4321c995f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.003708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876d2d1-24b8-492e-9272-b4321c995f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.003989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876d2d1-24b8-492e-9272-b4321c995f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.004094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b17353f5-0c7c-4416-b571-5bb8f185a6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:35.004576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876d2d1-24b8-492e-9272-b4321c995f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.006178 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876d2d1-24b8-492e-9272-b4321c995f76 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.011982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 773/1561] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:35.013273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0883ef2-dadd-4266-8cfd-8dccdeea59a2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.013426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0883ef2-dadd-4266-8cfd-8dccdeea59a2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.013700 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0883ef2-dadd-4266-8cfd-8dccdeea59a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.017356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 789/1562] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:35.021313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0fa88-4e88-49de-ad5c-8d89fb1d6342 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.021605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0fa88-4e88-49de-ad5c-8d89fb1d6342 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.021881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0fa88-4e88-49de-ad5c-8d89fb1d6342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.033505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e0883ef2-dadd-4266-8cfd-8dccdeea59a2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:35.034713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e0883ef2-dadd-4266-8cfd-8dccdeea59a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:35.041130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-acc0fa88-4e88-49de-ad5c-8d89fb1d6342 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:35.042297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-acc0fa88-4e88-49de-ad5c-8d89fb1d6342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:35.042873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 774/1563] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:35.043456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0883ef2-dadd-4266-8cfd-8dccdeea59a2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.052627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0883ef2-dadd-4266-8cfd-8dccdeea59a2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.053797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 790/1564] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:35.055008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743336b9-90fe-4e39-aeef-272cca7dc6e3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.055309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743336b9-90fe-4e39-aeef-272cca7dc6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.055562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743336b9-90fe-4e39-aeef-272cca7dc6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.057408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce8ef8-6fbb-445b-a1ca-3c54b6b18f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.058004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce8ef8-6fbb-445b-a1ca-3c54b6b18f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.058199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce8ef8-6fbb-445b-a1ca-3c54b6b18f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.058432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce8ef8-6fbb-445b-a1ca-3c54b6b18f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.059048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce8ef8-6fbb-445b-a1ca-3c54b6b18f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.060961 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce8ef8-6fbb-445b-a1ca-3c54b6b18f41 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.075384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-743336b9-90fe-4e39-aeef-272cca7dc6e3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:35.076627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-743336b9-90fe-4e39-aeef-272cca7dc6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:35.089100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 791/1565] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:35.089100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4a608-58ea-4080-87dd-ca106b8dd4b5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.089100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4a608-58ea-4080-87dd-ca106b8dd4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.089100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4a608-58ea-4080-87dd-ca106b8dd4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.093436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 775/1566] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.094154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce8ef8-6fbb-445b-a1ca-3c54b6b18f41 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.107476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2eb4a608-58ea-4080-87dd-ca106b8dd4b5 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:35.108931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2eb4a608-58ea-4080-87dd-ca106b8dd4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:35.113722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:35.162787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce8ef8-6fbb-445b-a1ca-3c54b6b18f41 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.167834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806d7ef-1593-42d0-b151-98ab33aad6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.168440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806d7ef-1593-42d0-b151-98ab33aad6fa admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.168723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806d7ef-1593-42d0-b151-98ab33aad6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.168997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1806d7ef-1593-42d0-b151-98ab33aad6fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.169653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806d7ef-1593-42d0-b151-98ab33aad6fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:35.169947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806d7ef-1593-42d0-b151-98ab33aad6fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.171598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1806d7ef-1593-42d0-b151-98ab33aad6fa admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.173202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 792/1567] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:35.174156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4a608-58ea-4080-87dd-ca106b8dd4b5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.183852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 776/1568] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:35.184963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806d7ef-1593-42d0-b151-98ab33aad6fa admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.185654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4a608-58ea-4080-87dd-ca106b8dd4b5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.190730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.191493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.191807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.192087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.192727 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:35.193028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.194048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:20:35.194048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 21 12:20:35.194048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: Sep 21 12:20:35.194936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.195311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1806d7ef-1593-42d0-b151-98ab33aad6fa admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.199930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2db20-f375-4cd1-b1fd-46f4c21f45af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.200524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2db20-f375-4cd1-b1fd-46f4c21f45af admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.200802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2db20-f375-4cd1-b1fd-46f4c21f45af admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.201073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2db20-f375-4cd1-b1fd-46f4c21f45af admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.201705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2db20-f375-4cd1-b1fd-46f4c21f45af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.203872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2db20-f375-4cd1-b1fd-46f4c21f45af admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.207307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 793/1569] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:35.208189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.214685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 777/1570] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:35.215412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2db20-f375-4cd1-b1fd-46f4c21f45af admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.225276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2db20-f375-4cd1-b1fd-46f4c21f45af admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.225734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf865d-4879-43a9-8719-cfef892b2262 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.230037 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596979-55d4-4933-8145-d1530abd9ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.230631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596979-55d4-4933-8145-d1530abd9ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.230914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596979-55d4-4933-8145-d1530abd9ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.231020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5c749-88d8-47d4-8e39-ef4a1af0172f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.231325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596979-55d4-4933-8145-d1530abd9ab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.231549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5c749-88d8-47d4-8e39-ef4a1af0172f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.231825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5c749-88d8-47d4-8e39-ef4a1af0172f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.231929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc596979-55d4-4933-8145-d1530abd9ab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:35.232098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5c749-88d8-47d4-8e39-ef4a1af0172f admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.232199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc596979-55d4-4933-8145-d1530abd9ab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.232733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5c749-88d8-47d4-8e39-ef4a1af0172f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:35.233033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5c749-88d8-47d4-8e39-ef4a1af0172f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.233759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc596979-55d4-4933-8145-d1530abd9ab7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.234652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5c749-88d8-47d4-8e39-ef4a1af0172f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.246189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 778/1571] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:35.247006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596979-55d4-4933-8145-d1530abd9ab7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.256366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 794/1572] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:35.256366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5c749-88d8-47d4-8e39-ef4a1af0172f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.261176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596979-55d4-4933-8145-d1530abd9ab7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.263684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5c749-88d8-47d4-8e39-ef4a1af0172f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.266496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4cbb5-c00f-4ac0-82c9-cb41041eb490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.267099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4cbb5-c00f-4ac0-82c9-cb41041eb490 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.267371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4cbb5-c00f-4ac0-82c9-cb41041eb490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.267645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4cbb5-c00f-4ac0-82c9-cb41041eb490 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.268319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4cbb5-c00f-4ac0-82c9-cb41041eb490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:35.268622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4cbb5-c00f-4ac0-82c9-cb41041eb490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.269023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c39b3-03eb-4744-9cc0-5daeb3d1c77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.269621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c39b3-03eb-4744-9cc0-5daeb3d1c77a admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.269899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c39b3-03eb-4744-9cc0-5daeb3d1c77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.270167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c39b3-03eb-4744-9cc0-5daeb3d1c77a admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.270277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4cbb5-c00f-4ac0-82c9-cb41041eb490 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.270790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c39b3-03eb-4744-9cc0-5daeb3d1c77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.272758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c39b3-03eb-4744-9cc0-5daeb3d1c77a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.282613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 779/1573] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:35.283427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4cbb5-c00f-4ac0-82c9-cb41041eb490 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.293923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4cbb5-c00f-4ac0-82c9-cb41041eb490 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.295446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 795/1574] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/users => generated 292 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.296304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c39b3-03eb-4744-9cc0-5daeb3d1c77a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.299800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f9871-8b24-43e1-84ea-c1908d692842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.300455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f9871-8b24-43e1-84ea-c1908d692842 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.300727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f9871-8b24-43e1-84ea-c1908d692842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.300996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f9871-8b24-43e1-84ea-c1908d692842 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.301639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f9871-8b24-43e1-84ea-c1908d692842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:35.301962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f9871-8b24-43e1-84ea-c1908d692842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.303571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f9871-8b24-43e1-84ea-c1908d692842 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.306050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c39b3-03eb-4744-9cc0-5daeb3d1c77a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.310626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8139818-3ed1-469c-a21d-82ff00d31eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.311216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8139818-3ed1-469c-a21d-82ff00d31eeb admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.311499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8139818-3ed1-469c-a21d-82ff00d31eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.312319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8139818-3ed1-469c-a21d-82ff00d31eeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.313085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8139818-3ed1-469c-a21d-82ff00d31eeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.314969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8139818-3ed1-469c-a21d-82ff00d31eeb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.316365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 780/1575] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:35.316997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f9871-8b24-43e1-84ea-c1908d692842 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.326726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f9871-8b24-43e1-84ea-c1908d692842 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.332286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd38f06-39a9-48fc-9e7f-1d5bd868a89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.332919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd38f06-39a9-48fc-9e7f-1d5bd868a89c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.333470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd38f06-39a9-48fc-9e7f-1d5bd868a89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.333763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd38f06-39a9-48fc-9e7f-1d5bd868a89c admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.334460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd38f06-39a9-48fc-9e7f-1d5bd868a89c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.336613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd38f06-39a9-48fc-9e7f-1d5bd868a89c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.345046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 796/1576] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.345877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8139818-3ed1-469c-a21d-82ff00d31eeb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.349126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:35.371947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 781/1577] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.371947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd38f06-39a9-48fc-9e7f-1d5bd868a89c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.411703 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8139818-3ed1-469c-a21d-82ff00d31eeb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.433732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd38f06-39a9-48fc-9e7f-1d5bd868a89c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.439312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5319e6c-52d6-43a7-9990-135c2b5f7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.439948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5319e6c-52d6-43a7-9990-135c2b5f7a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.440234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5319e6c-52d6-43a7-9990-135c2b5f7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.440506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5319e6c-52d6-43a7-9990-135c2b5f7a6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.441131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5319e6c-52d6-43a7-9990-135c2b5f7a6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.443036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5319e6c-52d6-43a7-9990-135c2b5f7a6e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.462728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:35.471926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 782/1578] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.473229 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5319e6c-52d6-43a7-9990-135c2b5f7a6e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.475350 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7546e-5f16-4e24-82be-3ad7392db6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.476701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7546e-5f16-4e24-82be-3ad7392db6af admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.477242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7546e-5f16-4e24-82be-3ad7392db6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.477242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7546e-5f16-4e24-82be-3ad7392db6af admin admin] PATH_INFO: `/v3/projects/cf8d461fc9c14568b687b373990b17c8/users/5f1ea4662764421f8655851d1a937f57/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.494050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7546e-5f16-4e24-82be-3ad7392db6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8d461fc9c14568b687b373990b17c8, user_id=5f1ea4662764421f8655851d1a937f57, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.497331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a7546e-5f16-4e24-82be-3ad7392db6af admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.511346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 797/1579] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:35 2022] PUT /identity/v3/projects/cf8d461fc9c14568b687b373990b17c8/users/5f1ea4662764421f8655851d1a937f57/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:35.512295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7546e-5f16-4e24-82be-3ad7392db6af admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.552559 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5319e6c-52d6-43a7-9990-135c2b5f7a6e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.581951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a7546e-5f16-4e24-82be-3ad7392db6af admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.587060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabb07f-c7f0-4852-9c5a-d229023d5939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.587673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabb07f-c7f0-4852-9c5a-d229023d5939 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.587984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabb07f-c7f0-4852-9c5a-d229023d5939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.588263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabb07f-c7f0-4852-9c5a-d229023d5939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.588892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabb07f-c7f0-4852-9c5a-d229023d5939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.590794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabb07f-c7f0-4852-9c5a-d229023d5939 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.612199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663b5a8-14d8-4ef3-84b3-c90ed9af7419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.612881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663b5a8-14d8-4ef3-84b3-c90ed9af7419 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.613230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663b5a8-14d8-4ef3-84b3-c90ed9af7419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.613515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663b5a8-14d8-4ef3-84b3-c90ed9af7419 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.614165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663b5a8-14d8-4ef3-84b3-c90ed9af7419 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.616130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663b5a8-14d8-4ef3-84b3-c90ed9af7419 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.619206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 798/1580] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.620126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabb07f-c7f0-4852-9c5a-d229023d5939 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.647617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 783/1581] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/projects => generated 389 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.648516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663b5a8-14d8-4ef3-84b3-c90ed9af7419 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.657947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:35.689190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabb07f-c7f0-4852-9c5a-d229023d5939 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.713365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663b5a8-14d8-4ef3-84b3-c90ed9af7419 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.718101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8f8f0-d039-443b-af80-3416609bd74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.718701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8f8f0-d039-443b-af80-3416609bd74d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.718981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8f8f0-d039-443b-af80-3416609bd74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.719411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8f8f0-d039-443b-af80-3416609bd74d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.720073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8f8f0-d039-443b-af80-3416609bd74d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.722430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8f8f0-d039-443b-af80-3416609bd74d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.747578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb8ce3-66c0-4059-8df8-01ab77cfcc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.748517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb8ce3-66c0-4059-8df8-01ab77cfcc9b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.748807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb8ce3-66c0-4059-8df8-01ab77cfcc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.749084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb8ce3-66c0-4059-8df8-01ab77cfcc9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.750438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb8ce3-66c0-4059-8df8-01ab77cfcc9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.752071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb8ce3-66c0-4059-8df8-01ab77cfcc9b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.752799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 784/1582] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.753678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8f8f0-d039-443b-af80-3416609bd74d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.765668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 799/1583] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:35.765668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb8ce3-66c0-4059-8df8-01ab77cfcc9b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.768657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:35.775565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb8ce3-66c0-4059-8df8-01ab77cfcc9b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.819075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8f8f0-d039-443b-af80-3416609bd74d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.823742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efa8c1-fb18-4394-8f58-821567069604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.824399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efa8c1-fb18-4394-8f58-821567069604 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.824682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efa8c1-fb18-4394-8f58-821567069604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.824972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24efa8c1-fb18-4394-8f58-821567069604 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.825582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efa8c1-fb18-4394-8f58-821567069604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.827571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24efa8c1-fb18-4394-8f58-821567069604 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.834431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe057f48-6544-4606-b970-6c2729469a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.835122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057f48-6544-4606-b970-6c2729469a9e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.835500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057f48-6544-4606-b970-6c2729469a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.835867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe057f48-6544-4606-b970-6c2729469a9e admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.836559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe057f48-6544-4606-b970-6c2729469a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.838540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe057f48-6544-4606-b970-6c2729469a9e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.857644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 785/1584] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/users => generated 301 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.858495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efa8c1-fb18-4394-8f58-821567069604 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.864995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 800/1585] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.866233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2887db-9717-4d6c-9d6f-cdca965c8b09 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.866599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2887db-9717-4d6c-9d6f-cdca965c8b09 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.866951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2887db-9717-4d6c-9d6f-cdca965c8b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.873356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24efa8c1-fb18-4394-8f58-821567069604 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.880624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89a1cb-7ab6-4c02-bf44-92b61845529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.881245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89a1cb-7ab6-4c02-bf44-92b61845529e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.881523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89a1cb-7ab6-4c02-bf44-92b61845529e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.881799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89a1cb-7ab6-4c02-bf44-92b61845529e admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.882440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89a1cb-7ab6-4c02-bf44-92b61845529e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.884612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89a1cb-7ab6-4c02-bf44-92b61845529e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.896214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cb2887db-9717-4d6c-9d6f-cdca965c8b09 None None] MFA Rules not processed for user `5f1ea4662764421f8655851d1a937f57`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:35.897537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cb2887db-9717-4d6c-9d6f-cdca965c8b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:35.907939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 786/1586] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/users => generated 290 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.908798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89a1cb-7ab6-4c02-bf44-92b61845529e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.918228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89a1cb-7ab6-4c02-bf44-92b61845529e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.923491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d062b3d-b985-446a-909e-0ab3c087df57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.924114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d062b3d-b985-446a-909e-0ab3c087df57 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.924396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d062b3d-b985-446a-909e-0ab3c087df57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.924678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d062b3d-b985-446a-909e-0ab3c087df57 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.925295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d062b3d-b985-446a-909e-0ab3c087df57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.927202 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d062b3d-b985-446a-909e-0ab3c087df57 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.949118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 787/1587] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/users => generated 289 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.949834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d062b3d-b985-446a-909e-0ab3c087df57 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.959998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d062b3d-b985-446a-909e-0ab3c087df57 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.964729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f89b4-3c48-4f43-be31-efb139e4fbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.965331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f89b4-3c48-4f43-be31-efb139e4fbfe admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.965604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f89b4-3c48-4f43-be31-efb139e4fbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.965894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f89b4-3c48-4f43-be31-efb139e4fbfe admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.966548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f89b4-3c48-4f43-be31-efb139e4fbfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.968644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 801/1588] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:35.969646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2887db-9717-4d6c-9d6f-cdca965c8b09 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:35.972156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f89b4-3c48-4f43-be31-efb139e4fbfe admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.980577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2887db-9717-4d6c-9d6f-cdca965c8b09 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:35.985942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56e3d9-fbde-49fa-b953-30c562e9b776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:35.986421 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56e3d9-fbde-49fa-b953-30c562e9b776 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:35.986770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56e3d9-fbde-49fa-b953-30c562e9b776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:35.987043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56e3d9-fbde-49fa-b953-30c562e9b776 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:35.987787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56e3d9-fbde-49fa-b953-30c562e9b776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:35.989626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56e3d9-fbde-49fa-b953-30c562e9b776 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:35.996639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 788/1589] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/users => generated 295 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:35.997284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f89b4-3c48-4f43-be31-efb139e4fbfe admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.009086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f89b4-3c48-4f43-be31-efb139e4fbfe admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.012352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 802/1590] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:35 2022] POST /identity/v3/users => generated 302 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:36.012352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56e3d9-fbde-49fa-b953-30c562e9b776 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.017530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22972886-760f-4f5c-8184-e98bd86f6b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.018217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22972886-760f-4f5c-8184-e98bd86f6b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.018496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22972886-760f-4f5c-8184-e98bd86f6b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.018963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22972886-760f-4f5c-8184-e98bd86f6b26 admin admin] PATH_INFO: `/v3/projects/25a6a74e37f54c63a74860f4119aa43f/users/989594847219494db6b94ea95ca7ef22/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.023457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56e3d9-fbde-49fa-b953-30c562e9b776 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.028002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf24aa-2dcc-408f-8ba5-abf5c0600837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.028588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf24aa-2dcc-408f-8ba5-abf5c0600837 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.028865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf24aa-2dcc-408f-8ba5-abf5c0600837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.029130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf24aa-2dcc-408f-8ba5-abf5c0600837 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.029761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf24aa-2dcc-408f-8ba5-abf5c0600837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.032008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf24aa-2dcc-408f-8ba5-abf5c0600837 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.034320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22972886-760f-4f5c-8184-e98bd86f6b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a6a74e37f54c63a74860f4119aa43f, user_id=989594847219494db6b94ea95ca7ef22, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.035349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-22972886-760f-4f5c-8184-e98bd86f6b26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:20:36.035349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 21 12:20:36.035349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: Sep 21 12:20:36.037339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22972886-760f-4f5c-8184-e98bd86f6b26 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.043813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 803/1591] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.046990 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf24aa-2dcc-408f-8ba5-abf5c0600837 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.055933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf24aa-2dcc-408f-8ba5-abf5c0600837 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.058954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 789/1592] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:35 2022] PUT /identity/v3/projects/25a6a74e37f54c63a74860f4119aa43f/users/989594847219494db6b94ea95ca7ef22/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:36.058954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22972886-760f-4f5c-8184-e98bd86f6b26 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.068097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:36.107190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:36.117775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221d843-b0ae-4b71-9591-21789ba55d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.118385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221d843-b0ae-4b71-9591-21789ba55d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.118665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221d843-b0ae-4b71-9591-21789ba55d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.118936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221d843-b0ae-4b71-9591-21789ba55d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.119577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221d843-b0ae-4b71-9591-21789ba55d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.121224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221d843-b0ae-4b71-9591-21789ba55d93 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.127537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22972886-760f-4f5c-8184-e98bd86f6b26 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.132174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 804/1593] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.133050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221d843-b0ae-4b71-9591-21789ba55d93 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.138609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53c2cb-0976-485b-80fd-c2bc434f017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.139233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53c2cb-0976-485b-80fd-c2bc434f017d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.139517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53c2cb-0976-485b-80fd-c2bc434f017d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.139838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53c2cb-0976-485b-80fd-c2bc434f017d admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.141158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53c2cb-0976-485b-80fd-c2bc434f017d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.142774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53c2cb-0976-485b-80fd-c2bc434f017d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.143415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221d843-b0ae-4b71-9591-21789ba55d93 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.149206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3917d4-65f3-4401-b7ad-d339384a45cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.149824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3917d4-65f3-4401-b7ad-d339384a45cd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.150268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3917d4-65f3-4401-b7ad-d339384a45cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.150547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3917d4-65f3-4401-b7ad-d339384a45cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.151607 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3917d4-65f3-4401-b7ad-d339384a45cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.153803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3917d4-65f3-4401-b7ad-d339384a45cd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.157663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 790/1594] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.158409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53c2cb-0976-485b-80fd-c2bc434f017d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.168743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 805/1595] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.168743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3917d4-65f3-4401-b7ad-d339384a45cd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.179463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3917d4-65f3-4401-b7ad-d339384a45cd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.184226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f964de-87ad-4651-ac73-5e00932ab6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.184831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f964de-87ad-4651-ac73-5e00932ab6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.185129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f964de-87ad-4651-ac73-5e00932ab6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.185388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f964de-87ad-4651-ac73-5e00932ab6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.186057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f964de-87ad-4651-ac73-5e00932ab6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.188375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f964de-87ad-4651-ac73-5e00932ab6fc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.198867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 806/1596] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.199683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f964de-87ad-4651-ac73-5e00932ab6fc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.209709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f964de-87ad-4651-ac73-5e00932ab6fc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.214226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e3cad-f3d2-4202-9ce3-45bb5dd0d171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.214809 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e3cad-f3d2-4202-9ce3-45bb5dd0d171 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.215090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e3cad-f3d2-4202-9ce3-45bb5dd0d171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.215387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e3cad-f3d2-4202-9ce3-45bb5dd0d171 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.216021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e3cad-f3d2-4202-9ce3-45bb5dd0d171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.217758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e3cad-f3d2-4202-9ce3-45bb5dd0d171 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.218822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53c2cb-0976-485b-80fd-c2bc434f017d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.223254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d70c58-3a1d-4256-b721-6497bfea9027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.224354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d70c58-3a1d-4256-b721-6497bfea9027 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.224656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d70c58-3a1d-4256-b721-6497bfea9027 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.224936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d70c58-3a1d-4256-b721-6497bfea9027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.231776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 807/1597] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.231776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e3cad-f3d2-4202-9ce3-45bb5dd0d171 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.242337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e3cad-f3d2-4202-9ce3-45bb5dd0d171 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.246909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884265f3-8fc7-4477-afb4-d77fd77e5caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.247500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884265f3-8fc7-4477-afb4-d77fd77e5caf admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.247799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884265f3-8fc7-4477-afb4-d77fd77e5caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.248075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884265f3-8fc7-4477-afb4-d77fd77e5caf admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.248708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884265f3-8fc7-4477-afb4-d77fd77e5caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.250837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884265f3-8fc7-4477-afb4-d77fd77e5caf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.260618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 808/1598] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.261499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884265f3-8fc7-4477-afb4-d77fd77e5caf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.271828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884265f3-8fc7-4477-afb4-d77fd77e5caf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.276895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f225347-6bdb-4049-bf66-36cf51e48919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.277511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f225347-6bdb-4049-bf66-36cf51e48919 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.277796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f225347-6bdb-4049-bf66-36cf51e48919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.278071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f225347-6bdb-4049-bf66-36cf51e48919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.278716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f225347-6bdb-4049-bf66-36cf51e48919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.280349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f225347-6bdb-4049-bf66-36cf51e48919 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.289305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d70c58-3a1d-4256-b721-6497bfea9027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.290731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d70c58-3a1d-4256-b721-6497bfea9027 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.290951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 809/1599] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.292708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f225347-6bdb-4049-bf66-36cf51e48919 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.300595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 791/1600] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:36.301465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d70c58-3a1d-4256-b721-6497bfea9027 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.303284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f225347-6bdb-4049-bf66-36cf51e48919 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.309504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf060dde-53a2-4443-9d59-e236bf242a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.310000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf060dde-53a2-4443-9d59-e236bf242a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.310308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf060dde-53a2-4443-9d59-e236bf242a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.310554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf060dde-53a2-4443-9d59-e236bf242a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.311193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf060dde-53a2-4443-9d59-e236bf242a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.311689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d70c58-3a1d-4256-b721-6497bfea9027 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.313477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf060dde-53a2-4443-9d59-e236bf242a8a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.317800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ceed2d-e0bd-4c6e-9633-58b6853c9528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.318527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceed2d-e0bd-4c6e-9633-58b6853c9528 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.318810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceed2d-e0bd-4c6e-9633-58b6853c9528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.319085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ceed2d-e0bd-4c6e-9633-58b6853c9528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.319715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ceed2d-e0bd-4c6e-9633-58b6853c9528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.321340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ceed2d-e0bd-4c6e-9633-58b6853c9528 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.325438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 810/1601] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.326768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf060dde-53a2-4443-9d59-e236bf242a8a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.334060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 792/1602] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.334920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ceed2d-e0bd-4c6e-9633-58b6853c9528 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.342178 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf060dde-53a2-4443-9d59-e236bf242a8a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.345037 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ceed2d-e0bd-4c6e-9633-58b6853c9528 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.349606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fab2df-b772-4e74-8ad4-71cbcfdfec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.350256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fab2df-b772-4e74-8ad4-71cbcfdfec42 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.350556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fab2df-b772-4e74-8ad4-71cbcfdfec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.350830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fab2df-b772-4e74-8ad4-71cbcfdfec42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.351475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fab2df-b772-4e74-8ad4-71cbcfdfec42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.352337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60dad5-07c5-4fa6-8b71-4ae4b621262a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.353136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fab2df-b772-4e74-8ad4-71cbcfdfec42 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.353890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60dad5-07c5-4fa6-8b71-4ae4b621262a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.354583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60dad5-07c5-4fa6-8b71-4ae4b621262a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.354754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60dad5-07c5-4fa6-8b71-4ae4b621262a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.355750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60dad5-07c5-4fa6-8b71-4ae4b621262a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.357037 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60dad5-07c5-4fa6-8b71-4ae4b621262a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.364564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 811/1603] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.365746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fab2df-b772-4e74-8ad4-71cbcfdfec42 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.371822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 793/1604] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.371822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60dad5-07c5-4fa6-8b71-4ae4b621262a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.377800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fab2df-b772-4e74-8ad4-71cbcfdfec42 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.382646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b7994-535d-45fc-82e2-03ec7a05057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.382866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60dad5-07c5-4fa6-8b71-4ae4b621262a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.383238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b7994-535d-45fc-82e2-03ec7a05057a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.383587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b7994-535d-45fc-82e2-03ec7a05057a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.383806 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b7994-535d-45fc-82e2-03ec7a05057a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.384461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b7994-535d-45fc-82e2-03ec7a05057a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.386006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b7994-535d-45fc-82e2-03ec7a05057a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.388898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22fcb0-d544-4a7f-90d8-138c2b5715f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.389686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22fcb0-d544-4a7f-90d8-138c2b5715f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.390550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22fcb0-d544-4a7f-90d8-138c2b5715f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.390824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22fcb0-d544-4a7f-90d8-138c2b5715f4 admin admin] PATH_INFO: `/v3/projects/bfe885783173412289ca18fb7cdc6bed/users/3ff1bb4a99444649bacccded0db5adbc/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.397522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 812/1605] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:36.400988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b7994-535d-45fc-82e2-03ec7a05057a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.409166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22fcb0-d544-4a7f-90d8-138c2b5715f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe885783173412289ca18fb7cdc6bed, user_id=3ff1bb4a99444649bacccded0db5adbc, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.411958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22fcb0-d544-4a7f-90d8-138c2b5715f4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.415847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b7994-535d-45fc-82e2-03ec7a05057a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.422454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fb233-2788-4d42-aa54-9cb4b7097525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.423275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fb233-2788-4d42-aa54-9cb4b7097525 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.423592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fb233-2788-4d42-aa54-9cb4b7097525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.423958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fb233-2788-4d42-aa54-9cb4b7097525 admin admin] PATH_INFO: `/v3/projects/25a6a74e37f54c63a74860f4119aa43f/users/989594847219494db6b94ea95ca7ef22/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.426108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fb233-2788-4d42-aa54-9cb4b7097525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a6a74e37f54c63a74860f4119aa43f, user_id=989594847219494db6b94ea95ca7ef22, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.428321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 794/1606] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:36 2022] PUT /identity/v3/projects/bfe885783173412289ca18fb7cdc6bed/users/3ff1bb4a99444649bacccded0db5adbc/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:36.428914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7fb233-2788-4d42-aa54-9cb4b7097525 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.429270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22fcb0-d544-4a7f-90d8-138c2b5715f4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.441071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 813/1607] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:36 2022] PUT /identity/v3/projects/25a6a74e37f54c63a74860f4119aa43f/users/989594847219494db6b94ea95ca7ef22/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:36.441901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fb233-2788-4d42-aa54-9cb4b7097525 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.445617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:36.515019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fb233-2788-4d42-aa54-9cb4b7097525 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.516235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22fcb0-d544-4a7f-90d8-138c2b5715f4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.519785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043545f-0c71-493a-a0c7-09733c57393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.520458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043545f-0c71-493a-a0c7-09733c57393b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.520739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043545f-0c71-493a-a0c7-09733c57393b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.521017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043545f-0c71-493a-a0c7-09733c57393b admin admin] PATH_INFO: `/v3/projects/74d5d713723f4ea5a8d1e72627b80c47/users/2ab0433dd3864867b03c5dd6407aa388/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.521875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4c504-acb6-48e7-bc1a-fa30355fad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.522926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4c504-acb6-48e7-bc1a-fa30355fad14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.523218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4c504-acb6-48e7-bc1a-fa30355fad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.523490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4c504-acb6-48e7-bc1a-fa30355fad14 admin admin] PATH_INFO: `/v3/projects/1ca32cb607f84222a9f547700f6c5fe4/users/df8530247db44cf09912f1d2d15c1559/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.538502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043545f-0c71-493a-a0c7-09733c57393b admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d5d713723f4ea5a8d1e72627b80c47, user_id=2ab0433dd3864867b03c5dd6407aa388, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.540834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4c504-acb6-48e7-bc1a-fa30355fad14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ca32cb607f84222a9f547700f6c5fe4, user_id=df8530247db44cf09912f1d2d15c1559, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.542325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043545f-0c71-493a-a0c7-09733c57393b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.543614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4c504-acb6-48e7-bc1a-fa30355fad14 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.562172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 814/1608] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:36 2022] PUT /identity/v3/projects/74d5d713723f4ea5a8d1e72627b80c47/users/2ab0433dd3864867b03c5dd6407aa388/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:36.562172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043545f-0c71-493a-a0c7-09733c57393b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.562477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 795/1609] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:36 2022] PUT /identity/v3/projects/1ca32cb607f84222a9f547700f6c5fe4/users/df8530247db44cf09912f1d2d15c1559/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:36.562477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4c504-acb6-48e7-bc1a-fa30355fad14 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.609180 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:36.641704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4c504-acb6-48e7-bc1a-fa30355fad14 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.645830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043545f-0c71-493a-a0c7-09733c57393b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.646919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f97578-1d6f-46c8-a3c3-07a3f496ce5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.647588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f97578-1d6f-46c8-a3c3-07a3f496ce5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.647905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f97578-1d6f-46c8-a3c3-07a3f496ce5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.648182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f97578-1d6f-46c8-a3c3-07a3f496ce5c admin admin] PATH_INFO: `/v3/projects/5e6ef4f3de1943ffbde82d3f0a19542f/users/c950eb00b9474b29b86d24983e3b1583/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.653336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e411678-aa7c-4ea4-9478-a833fa5d258a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.654162 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e411678-aa7c-4ea4-9478-a833fa5d258a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.654429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e411678-aa7c-4ea4-9478-a833fa5d258a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.654711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e411678-aa7c-4ea4-9478-a833fa5d258a admin admin] PATH_INFO: `/v3/projects/9340222cfe154efe8a6064b197d54499/users/8790c086dcaa4da5b739bcd02d30b4c0/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.665505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f97578-1d6f-46c8-a3c3-07a3f496ce5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6ef4f3de1943ffbde82d3f0a19542f, user_id=c950eb00b9474b29b86d24983e3b1583, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.668256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f97578-1d6f-46c8-a3c3-07a3f496ce5c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.671833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e411678-aa7c-4ea4-9478-a833fa5d258a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9340222cfe154efe8a6064b197d54499, user_id=8790c086dcaa4da5b739bcd02d30b4c0, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.674744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e411678-aa7c-4ea4-9478-a833fa5d258a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.684954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 796/1610] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:36 2022] PUT /identity/v3/projects/5e6ef4f3de1943ffbde82d3f0a19542f/users/c950eb00b9474b29b86d24983e3b1583/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:36.685825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f97578-1d6f-46c8-a3c3-07a3f496ce5c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:36.692592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 815/1611] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:36 2022] PUT /identity/v3/projects/9340222cfe154efe8a6064b197d54499/users/8790c086dcaa4da5b739bcd02d30b4c0/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:36.694242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a73476-d2fb-425f-a944-8ab96bd402c0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.694590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a73476-d2fb-425f-a944-8ab96bd402c0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.694952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a73476-d2fb-425f-a944-8ab96bd402c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.715699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f0a73476-d2fb-425f-a944-8ab96bd402c0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:36.716957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f0a73476-d2fb-425f-a944-8ab96bd402c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:36.772921 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f97578-1d6f-46c8-a3c3-07a3f496ce5c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:36.784707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 816/1612] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:36.786057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e09e-b6bd-43b3-8fd8-63781c77631d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.786324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e09e-b6bd-43b3-8fd8-63781c77631d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.786600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3e09e-b6bd-43b3-8fd8-63781c77631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.795159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:36.823561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0bb3e09e-b6bd-43b3-8fd8-63781c77631d None None] MFA Rules not processed for user `989594847219494db6b94ea95ca7ef22`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:36.824831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0bb3e09e-b6bd-43b3-8fd8-63781c77631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:36.841495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a374e0-19c5-45ca-91ca-860414dcc766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:36.842166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a374e0-19c5-45ca-91ca-860414dcc766 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.842442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a374e0-19c5-45ca-91ca-860414dcc766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.842714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a374e0-19c5-45ca-91ca-860414dcc766 admin admin] PATH_INFO: `/v3/projects/6b33841bf7d54fbea51c7d4ea55b7d50/users/36dce8fd1af4414a81312d673b48a989/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.863493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a374e0-19c5-45ca-91ca-860414dcc766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b33841bf7d54fbea51c7d4ea55b7d50, user_id=36dce8fd1af4414a81312d673b48a989, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:36.867560 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a374e0-19c5-45ca-91ca-860414dcc766 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:36.895844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 797/1613] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:36 2022] PUT /identity/v3/projects/6b33841bf7d54fbea51c7d4ea55b7d50/users/36dce8fd1af4414a81312d673b48a989/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:36.897599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d303-304c-404a-8785-bdb925ab7f3b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.898101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d303-304c-404a-8785-bdb925ab7f3b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.898685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9d303-304c-404a-8785-bdb925ab7f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.930437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 817/1614] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:36.930437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840406a-7d88-4e6d-bd46-b0381c8fb973 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:36.930437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840406a-7d88-4e6d-bd46-b0381c8fb973 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:36.930437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840406a-7d88-4e6d-bd46-b0381c8fb973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:36.930814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b6c9d303-304c-404a-8785-bdb925ab7f3b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:36.931549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b6c9d303-304c-404a-8785-bdb925ab7f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:36.971130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5840406a-7d88-4e6d-bd46-b0381c8fb973 None None] MFA Rules not processed for user `df8530247db44cf09912f1d2d15c1559`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:36.973018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5840406a-7d88-4e6d-bd46-b0381c8fb973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:37.006196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 798/1615] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:37.007046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:37.008181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3347-60e9-4780-9815-6275586e2c95 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.009186 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3347-60e9-4780-9815-6275586e2c95 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.009993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3347-60e9-4780-9815-6275586e2c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.033184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3e4a3347-60e9-4780-9815-6275586e2c95 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:37.034654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3e4a3347-60e9-4780-9815-6275586e2c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:37.043629 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 799/1616] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:37.048596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99205e70-4dde-4066-a945-4abf1622138a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.048891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99205e70-4dde-4066-a945-4abf1622138a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.049163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99205e70-4dde-4066-a945-4abf1622138a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.051053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 818/1617] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:37.052313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840406a-7d88-4e6d-bd46-b0381c8fb973 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.078533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-99205e70-4dde-4066-a945-4abf1622138a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:37.080062 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-99205e70-4dde-4066-a945-4abf1622138a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:37.089628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 800/1618] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:37.091734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d8575-f952-4779-b5da-8101d396e083 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.091882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d8575-f952-4779-b5da-8101d396e083 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.092291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d8575-f952-4779-b5da-8101d396e083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.126780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-753d8575-f952-4779-b5da-8101d396e083 None None] MFA Rules not processed for user `36dce8fd1af4414a81312d673b48a989`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:37.128237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-753d8575-f952-4779-b5da-8101d396e083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:37.136622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840406a-7d88-4e6d-bd46-b0381c8fb973 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.140469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4cee9-2b12-421f-abb3-c74bb10c00a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.141081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4cee9-2b12-421f-abb3-c74bb10c00a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.141360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4cee9-2b12-421f-abb3-c74bb10c00a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.141633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4cee9-2b12-421f-abb3-c74bb10c00a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.150973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4cee9-2b12-421f-abb3-c74bb10c00a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.152442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4cee9-2b12-421f-abb3-c74bb10c00a4 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.162886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 819/1619] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:37.163690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4cee9-2b12-421f-abb3-c74bb10c00a4 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.206602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 801/1620] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:37.207481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d8575-f952-4779-b5da-8101d396e083 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.218548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d8575-f952-4779-b5da-8101d396e083 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.222686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d47eab-64dc-461a-a4c8-f8497fccbe45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.224962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d47eab-64dc-461a-a4c8-f8497fccbe45 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.225253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d47eab-64dc-461a-a4c8-f8497fccbe45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.225536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d47eab-64dc-461a-a4c8-f8497fccbe45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.237846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d47eab-64dc-461a-a4c8-f8497fccbe45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.239329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d47eab-64dc-461a-a4c8-f8497fccbe45 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.239897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4cee9-2b12-421f-abb3-c74bb10c00a4 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.244235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f619065-9c32-4f38-b6c9-60ee7bdcb54a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.245443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f619065-9c32-4f38-b6c9-60ee7bdcb54a service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.245443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f619065-9c32-4f38-b6c9-60ee7bdcb54a service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.245443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f619065-9c32-4f38-b6c9-60ee7bdcb54a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.250918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 802/1621] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:37.251993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d47eab-64dc-461a-a4c8-f8497fccbe45 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.260283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d47eab-64dc-461a-a4c8-f8497fccbe45 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.264474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d3cf5-99ec-4981-8a1b-20adf3534078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.265169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d3cf5-99ec-4981-8a1b-20adf3534078 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.265450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d3cf5-99ec-4981-8a1b-20adf3534078 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.265724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d3cf5-99ec-4981-8a1b-20adf3534078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.278609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d3cf5-99ec-4981-8a1b-20adf3534078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.280138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d3cf5-99ec-4981-8a1b-20adf3534078 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.286324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:37.295176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 803/1622] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:37.295923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d3cf5-99ec-4981-8a1b-20adf3534078 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.306288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d3cf5-99ec-4981-8a1b-20adf3534078 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.311379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa8034-a7f6-4612-aae1-97e568913e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.313009 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa8034-a7f6-4612-aae1-97e568913e1f service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.313361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa8034-a7f6-4612-aae1-97e568913e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.313635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa8034-a7f6-4612-aae1-97e568913e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.323314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f619065-9c32-4f38-b6c9-60ee7bdcb54a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.324574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa8034-a7f6-4612-aae1-97e568913e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.325328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f619065-9c32-4f38-b6c9-60ee7bdcb54a service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.326708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa8034-a7f6-4612-aae1-97e568913e1f service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.337738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 804/1623] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:37.338910 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 820/1624] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:37.339966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f619065-9c32-4f38-b6c9-60ee7bdcb54a service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.343814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3e4fd-7d52-45b0-96d5-ed216f00d12f None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.344675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3e4fd-7d52-45b0-96d5-ed216f00d12f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.345003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d3e4fd-7d52-45b0-96d5-ed216f00d12f None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.347153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 805/1625] 10.222.0.71 () {54 vars in 965 bytes} [Wed Sep 21 12:20:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:20:37.353363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d469fec-6f5c-4822-afa2-993a922af41e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.353595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f619065-9c32-4f38-b6c9-60ee7bdcb54a service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.353711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d469fec-6f5c-4822-afa2-993a922af41e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.354103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d469fec-6f5c-4822-afa2-993a922af41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.359713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd1875-c85d-4f52-9f2e-e07c0017790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.360412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd1875-c85d-4f52-9f2e-e07c0017790b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.360741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd1875-c85d-4f52-9f2e-e07c0017790b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.361023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd1875-c85d-4f52-9f2e-e07c0017790b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.361768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd1875-c85d-4f52-9f2e-e07c0017790b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.364168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd1875-c85d-4f52-9f2e-e07c0017790b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.392405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:37.402861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 821/1626] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:37 2022] POST /identity/v3/projects => generated 405 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:37.412545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8d469fec-6f5c-4822-afa2-993a922af41e None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:37.412737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd1875-c85d-4f52-9f2e-e07c0017790b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.412854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8d469fec-6f5c-4822-afa2-993a922af41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:37.518145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd1875-c85d-4f52-9f2e-e07c0017790b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.524190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2349b5c-1296-4eb1-9f39-787ce0b7fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.528049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2349b5c-1296-4eb1-9f39-787ce0b7fc8b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.528049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2349b5c-1296-4eb1-9f39-787ce0b7fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.528049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2349b5c-1296-4eb1-9f39-787ce0b7fc8b admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.528049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2349b5c-1296-4eb1-9f39-787ce0b7fc8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.528886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2349b5c-1296-4eb1-9f39-787ce0b7fc8b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.536207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 806/1627] 10.222.0.71 () {58 vars in 1062 bytes} [Wed Sep 21 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:37.555791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d469fec-6f5c-4822-afa2-993a922af41e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.559111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 822/1628] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:37 2022] POST /identity/v3/users => generated 309 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:37.563704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2349b5c-1296-4eb1-9f39-787ce0b7fc8b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.571460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d469fec-6f5c-4822-afa2-993a922af41e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.579170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2349b5c-1296-4eb1-9f39-787ce0b7fc8b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.589526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c93ffa-61b2-4e81-b010-d703ef02a2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.590344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c93ffa-61b2-4e81-b010-d703ef02a2db admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.590748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c93ffa-61b2-4e81-b010-d703ef02a2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.591176 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c93ffa-61b2-4e81-b010-d703ef02a2db admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.592567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c93ffa-61b2-4e81-b010-d703ef02a2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.594707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c93ffa-61b2-4e81-b010-d703ef02a2db admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.602533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8aa6b8-2fb3-4768-a395-c4913bffa151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.604113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8aa6b8-2fb3-4768-a395-c4913bffa151 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.604406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8aa6b8-2fb3-4768-a395-c4913bffa151 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.605288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8aa6b8-2fb3-4768-a395-c4913bffa151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.620148 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 823/1629] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:37.622890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c93ffa-61b2-4e81-b010-d703ef02a2db admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.636714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c93ffa-61b2-4e81-b010-d703ef02a2db admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.643865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b97cc-4caf-42e1-a2c8-7007139dccc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.644717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b97cc-4caf-42e1-a2c8-7007139dccc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.645003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b97cc-4caf-42e1-a2c8-7007139dccc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.645420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b97cc-4caf-42e1-a2c8-7007139dccc4 admin admin] PATH_INFO: `/v3/projects/e2aa4b291f4e4c8ba26e7c136b33b5b8/users/b79b75bbd3304975b0a75ad8c5bda9a8/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.665818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b97cc-4caf-42e1-a2c8-7007139dccc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2aa4b291f4e4c8ba26e7c136b33b5b8, user_id=b79b75bbd3304975b0a75ad8c5bda9a8, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.669784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b97cc-4caf-42e1-a2c8-7007139dccc4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.689180 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 824/1630] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:37 2022] PUT /identity/v3/projects/e2aa4b291f4e4c8ba26e7c136b33b5b8/users/b79b75bbd3304975b0a75ad8c5bda9a8/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:37.693229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b97cc-4caf-42e1-a2c8-7007139dccc4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.719836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8aa6b8-2fb3-4768-a395-c4913bffa151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.721655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8aa6b8-2fb3-4768-a395-c4913bffa151 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.734625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 807/1631] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:37.829511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b97cc-4caf-42e1-a2c8-7007139dccc4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.845005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4cf73-8983-421a-ab1f-a8c8d7f6c923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.845650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4cf73-8983-421a-ab1f-a8c8d7f6c923 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.845922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4cf73-8983-421a-ab1f-a8c8d7f6c923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.846199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4cf73-8983-421a-ab1f-a8c8d7f6c923 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.846881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4cf73-8983-421a-ab1f-a8c8d7f6c923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.848543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4cf73-8983-421a-ab1f-a8c8d7f6c923 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.873312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 825/1632] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:20:37.878467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8aa6b8-2fb3-4768-a395-c4913bffa151 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:37.910104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8aa6b8-2fb3-4768-a395-c4913bffa151 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:37.924798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81d00d-a66b-4e66-8418-4a505efaf794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:37.925556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81d00d-a66b-4e66-8418-4a505efaf794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.925841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81d00d-a66b-4e66-8418-4a505efaf794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.926120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81d00d-a66b-4e66-8418-4a505efaf794 admin admin] PATH_INFO: `/v3/projects/e2aa4b291f4e4c8ba26e7c136b33b5b8/users/b79b75bbd3304975b0a75ad8c5bda9a8/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:37.929996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81d00d-a66b-4e66-8418-4a505efaf794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2aa4b291f4e4c8ba26e7c136b33b5b8, user_id=b79b75bbd3304975b0a75ad8c5bda9a8, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:37.948938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81d00d-a66b-4e66-8418-4a505efaf794 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:37.971337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 808/1633] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:37 2022] PUT /identity/v3/projects/e2aa4b291f4e4c8ba26e7c136b33b5b8/users/b79b75bbd3304975b0a75ad8c5bda9a8/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:37.981374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246482f6-f48a-4960-af6c-432e464947b2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:37.981865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246482f6-f48a-4960-af6c-432e464947b2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:37.982440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246482f6-f48a-4960-af6c-432e464947b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:38.044248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-246482f6-f48a-4960-af6c-432e464947b2 None None] MFA Rules not processed for user `b79b75bbd3304975b0a75ad8c5bda9a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:38.045573 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-246482f6-f48a-4960-af6c-432e464947b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:38.055232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81d00d-a66b-4e66-8418-4a505efaf794 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:38.143936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:38.145069 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:38.170748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81d00d-a66b-4e66-8418-4a505efaf794 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:38.180307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:38.181085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:38.181434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:38.182187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] PATH_INFO: `/v3/users/c2211bc31f8f49f39623ee643e6d812d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:38.184169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2211bc31f8f49f39623ee643e6d812d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:38.185736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: WARNING py.warnings [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:20:38.185736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: warnings.warn(msg) Sep 21 12:20:38.185736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: Sep 21 12:20:38.189085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:38.194783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 826/1634] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:38.196122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc6e38-f238-4d48-ab0a-d1994aafb1ad None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:38.196416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc6e38-f238-4d48-ab0a-d1994aafb1ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:38.196692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc6e38-f238-4d48-ab0a-d1994aafb1ad None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:38.197966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 827/1635] 10.222.0.71 () {54 vars in 912 bytes} [Wed Sep 21 12:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:20:38.226777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97049f5d-6643-472e-8951-c67f8cf45539 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:38.227068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97049f5d-6643-472e-8951-c67f8cf45539 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:38.227351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97049f5d-6643-472e-8951-c67f8cf45539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:38.255224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-97049f5d-6643-472e-8951-c67f8cf45539 None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:38.256460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97049f5d-6643-472e-8951-c67f8cf45539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:38.286481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2211bc31f8f49f39623ee643e6d812d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:38.287425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2211bc31f8f49f39623ee643e6d812d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:38.298132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2211bc31f8f49f39623ee643e6d812d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:38.299286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2211bc31f8f49f39623ee643e6d812d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:38.315275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2211bc31f8f49f39623ee643e6d812d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:38.357607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 809/1636] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:38 2022] DELETE /identity/v3/users/c2211bc31f8f49f39623ee643e6d812d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:38.359275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:38.361264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:38.381328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 828/1637] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:20:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:38.383304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db5a01-193b-4e93-9e9b-9046fcbe9ab4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:38.383818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db5a01-193b-4e93-9e9b-9046fcbe9ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:38.384532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db5a01-193b-4e93-9e9b-9046fcbe9ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:38.464044 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-61db5a01-193b-4e93-9e9b-9046fcbe9ab4 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:38.465290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-61db5a01-193b-4e93-9e9b-9046fcbe9ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:38.499664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a2159-7c3f-41e7-9968-d1b3bb1490da admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:38.504224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e4ba-36d5-4178-9aab-0a54b2d1fe9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:38.505598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e4ba-36d5-4178-9aab-0a54b2d1fe9f service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:38.505888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e4ba-36d5-4178-9aab-0a54b2d1fe9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:38.506163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e4ba-36d5-4178-9aab-0a54b2d1fe9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:38.508418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9a46e4ba-36d5-4178-9aab-0a54b2d1fe9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:38.601453 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e4ba-36d5-4178-9aab-0a54b2d1fe9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:38.603261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e4ba-36d5-4178-9aab-0a54b2d1fe9f service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:38.621486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 829/1638] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:38.624362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db5a01-193b-4e93-9e9b-9046fcbe9ab4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:38.627349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-61db5a01-193b-4e93-9e9b-9046fcbe9ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:38.637319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 810/1639] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:38 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:38.641315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d23d5-0e51-42ca-9443-28804050872e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:38.641700 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d23d5-0e51-42ca-9443-28804050872e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:38.641985 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d23d5-0e51-42ca-9443-28804050872e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:38.660904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db5a01-193b-4e93-9e9b-9046fcbe9ab4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:38.679554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6aebeb-badf-4ba7-a3f3-62ea0f348cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:38.680382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6aebeb-badf-4ba7-a3f3-62ea0f348cac service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:38.680795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6aebeb-badf-4ba7-a3f3-62ea0f348cac service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:38.681186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6aebeb-badf-4ba7-a3f3-62ea0f348cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:38.716246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-280d23d5-0e51-42ca-9443-28804050872e None None] MFA Rules not processed for user `3ff1bb4a99444649bacccded0db5adbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:38.718561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-280d23d5-0e51-42ca-9443-28804050872e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:38.726889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:38.841876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:38.846473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6aebeb-badf-4ba7-a3f3-62ea0f348cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:38.847942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6aebeb-badf-4ba7-a3f3-62ea0f348cac service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:38.866508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 830/1640] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:38.873602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6aebeb-badf-4ba7-a3f3-62ea0f348cac service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:38.874675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 811/1641] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:38 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:38.876109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d23d5-0e51-42ca-9443-28804050872e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:38.878322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-280d23d5-0e51-42ca-9443-28804050872e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:38.878621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1f6aebeb-badf-4ba7-a3f3-62ea0f348cac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:38.897182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d23d5-0e51-42ca-9443-28804050872e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:38.903468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404542de-851d-4a11-8e1b-162ce192365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:38.904722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404542de-851d-4a11-8e1b-162ce192365b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:38.905235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404542de-851d-4a11-8e1b-162ce192365b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:38.906387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404542de-851d-4a11-8e1b-162ce192365b admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:38.907189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404542de-851d-4a11-8e1b-162ce192365b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:38.911307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404542de-851d-4a11-8e1b-162ce192365b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:38.996606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 812/1642] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:38 2022] POST /identity/v3/projects => generated 403 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:38.996606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404542de-851d-4a11-8e1b-162ce192365b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:39.001549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-404542de-851d-4a11-8e1b-162ce192365b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:39.019960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6aebeb-badf-4ba7-a3f3-62ea0f348cac service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:39.156266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404542de-851d-4a11-8e1b-162ce192365b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:39.165089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffed155-d194-489f-a766-d27b9b134e8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:39.166540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffed155-d194-489f-a766-d27b9b134e8f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.166837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffed155-d194-489f-a766-d27b9b134e8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.167092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffed155-d194-489f-a766-d27b9b134e8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.170608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:39.171880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.172134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.172406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] PATH_INFO: `/v3/users/36dce8fd1af4414a81312d673b48a989` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.176565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] RBAC: Authorizing `identity:delete_user(user_id=36dce8fd1af4414a81312d673b48a989)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:39.179648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:39.212248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffed155-d194-489f-a766-d27b9b134e8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:39.214575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffed155-d194-489f-a766-d27b9b134e8f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:39.246510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 831/1643] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:39.247559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffed155-d194-489f-a766-d27b9b134e8f service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:39.261173 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffed155-d194-489f-a766-d27b9b134e8f service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:39.267346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149b386-8798-4b75-b9b8-8772c42ce710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:39.268110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149b386-8798-4b75-b9b8-8772c42ce710 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.268463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149b386-8798-4b75-b9b8-8772c42ce710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.268782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149b386-8798-4b75-b9b8-8772c42ce710 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.269471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149b386-8798-4b75-b9b8-8772c42ce710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:39.271694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149b386-8798-4b75-b9b8-8772c42ce710 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:39.305358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 832/1644] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:39 2022] POST /identity/v3/users => generated 308 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:39.306972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149b386-8798-4b75-b9b8-8772c42ce710 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:39.311561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36dce8fd1af4414a81312d673b48a989'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:39.313353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36dce8fd1af4414a81312d673b48a989'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:39.324407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149b386-8798-4b75-b9b8-8772c42ce710 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:39.328283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36dce8fd1af4414a81312d673b48a989'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:39.329053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36dce8fd1af4414a81312d673b48a989'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:39.334552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305236fc-881e-45e2-920a-313701a5256b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:39.336187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305236fc-881e-45e2-920a-313701a5256b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.336537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305236fc-881e-45e2-920a-313701a5256b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.336850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305236fc-881e-45e2-920a-313701a5256b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.337771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305236fc-881e-45e2-920a-313701a5256b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:39.339843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305236fc-881e-45e2-920a-313701a5256b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:39.357745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-577e528c-c5df-4669-b2cd-14ffb1a0484c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36dce8fd1af4414a81312d673b48a989'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:39.360967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 833/1645] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:39.367708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305236fc-881e-45e2-920a-313701a5256b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:39.371735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-305236fc-881e-45e2-920a-313701a5256b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:39.395021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 813/1646] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:38 2022] DELETE /identity/v3/users/36dce8fd1af4414a81312d673b48a989 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:39.399339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae95b4-f9fa-48ee-a9a9-5023a03f7e66 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.400194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae95b4-f9fa-48ee-a9a9-5023a03f7e66 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.400828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae95b4-f9fa-48ee-a9a9-5023a03f7e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.420728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305236fc-881e-45e2-920a-313701a5256b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:39.429165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:39.482427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d1ae95b4-f9fa-48ee-a9a9-5023a03f7e66 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:39.484245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d1ae95b4-f9fa-48ee-a9a9-5023a03f7e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:39.596216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df45c7b-6fa0-4228-baaf-34742da012ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:39.596907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df45c7b-6fa0-4228-baaf-34742da012ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.597191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df45c7b-6fa0-4228-baaf-34742da012ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.597470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df45c7b-6fa0-4228-baaf-34742da012ae admin admin] PATH_INFO: `/v3/projects/aedd846e34fc471685daeec69f45b6cb/users/910d177b6f89421ca31e201fb82378f0/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.632124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df45c7b-6fa0-4228-baaf-34742da012ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=aedd846e34fc471685daeec69f45b6cb, user_id=910d177b6f89421ca31e201fb82378f0, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:39.636526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df45c7b-6fa0-4228-baaf-34742da012ae admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:39.668824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 834/1647] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:39 2022] PUT /identity/v3/projects/aedd846e34fc471685daeec69f45b6cb/users/910d177b6f89421ca31e201fb82378f0/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:39.673651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df45c7b-6fa0-4228-baaf-34742da012ae admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:39.675649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 814/1648] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:20:39.687925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae95b4-f9fa-48ee-a9a9-5023a03f7e66 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:39.690299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d1ae95b4-f9fa-48ee-a9a9-5023a03f7e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:39.691884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:39.835470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df45c7b-6fa0-4228-baaf-34742da012ae admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:39.850761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6aea9a-958a-40e9-a337-03d41f24cb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:39.851204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae95b4-f9fa-48ee-a9a9-5023a03f7e66 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:39.851966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aea9a-958a-40e9-a337-03d41f24cb32 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.853118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aea9a-958a-40e9-a337-03d41f24cb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.853857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aea9a-958a-40e9-a337-03d41f24cb32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.854823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6aea9a-958a-40e9-a337-03d41f24cb32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:39.860070 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6aea9a-958a-40e9-a337-03d41f24cb32 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:39.893129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 835/1649] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:20:39.893129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406fa1-3b2d-47ec-a141-51dfd5f42dad None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.893129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406fa1-3b2d-47ec-a141-51dfd5f42dad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.893129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406fa1-3b2d-47ec-a141-51dfd5f42dad None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.894049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 836/1650] 10.222.0.71 () {54 vars in 965 bytes} [Wed Sep 21 12:20:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:20:39.894934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d406fa1-3b2d-47ec-a141-51dfd5f42dad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:39.912870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d406fa1-3b2d-47ec-a141-51dfd5f42dad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:39.920338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda392c-6764-4f98-82e0-c6f5c486da95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:39.961289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda392c-6764-4f98-82e0-c6f5c486da95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.961289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda392c-6764-4f98-82e0-c6f5c486da95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.961289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda392c-6764-4f98-82e0-c6f5c486da95 admin admin] PATH_INFO: `/v3/projects/aedd846e34fc471685daeec69f45b6cb/users/910d177b6f89421ca31e201fb82378f0/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.961289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda392c-6764-4f98-82e0-c6f5c486da95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aedd846e34fc471685daeec69f45b6cb, user_id=910d177b6f89421ca31e201fb82378f0, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:39.961289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda392c-6764-4f98-82e0-c6f5c486da95 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:39.964981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 837/1651] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:39 2022] PUT /identity/v3/projects/aedd846e34fc471685daeec69f45b6cb/users/910d177b6f89421ca31e201fb82378f0/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:39.970691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fadd4-654d-479f-968d-4c8863ba94f4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.970691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fadd4-654d-479f-968d-4c8863ba94f4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.972991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015fadd4-654d-479f-968d-4c8863ba94f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.976511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00400dc-5019-4af5-b610-a1ce56553212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:39.977505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00400dc-5019-4af5-b610-a1ce56553212 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:39.978112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00400dc-5019-4af5-b610-a1ce56553212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:39.978628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00400dc-5019-4af5-b610-a1ce56553212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:39.992982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00400dc-5019-4af5-b610-a1ce56553212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:39.995604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00400dc-5019-4af5-b610-a1ce56553212 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:40.018730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 815/1652] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:40.020130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-015fadd4-654d-479f-968d-4c8863ba94f4 None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:40.021498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-015fadd4-654d-479f-968d-4c8863ba94f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:40.121001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 838/1653] 10.222.0.71 () {58 vars in 1062 bytes} [Wed Sep 21 12:20:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:40.126638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03375-ea9c-4588-ba23-e61c010f9dc7 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:40.126924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03375-ea9c-4588-ba23-e61c010f9dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:40.127184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03375-ea9c-4588-ba23-e61c010f9dc7 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:40.128735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 816/1654] 10.222.0.71 () {54 vars in 912 bytes} [Wed Sep 21 12:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:20:40.136902 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bd69e-11e0-4970-8651-1dbe0f4480f5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:40.137237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bd69e-11e0-4970-8651-1dbe0f4480f5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:40.137498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bd69e-11e0-4970-8651-1dbe0f4480f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:40.168416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-666bd69e-11e0-4970-8651-1dbe0f4480f5 None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:40.172004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-666bd69e-11e0-4970-8651-1dbe0f4480f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:40.191538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 839/1655] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:20:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:40.198385 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03375-ea9c-4588-ba23-e61c010f9dc7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:40.200310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b9e03375-ea9c-4588-ba23-e61c010f9dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:40.303209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03375-ea9c-4588-ba23-e61c010f9dc7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:40.308223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2604c2b-e60d-4a93-b09c-4e011d21bb01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:40.308950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2604c2b-e60d-4a93-b09c-4e011d21bb01 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:40.309224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2604c2b-e60d-4a93-b09c-4e011d21bb01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:40.309569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2604c2b-e60d-4a93-b09c-4e011d21bb01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:40.324344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2604c2b-e60d-4a93-b09c-4e011d21bb01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:40.326435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2604c2b-e60d-4a93-b09c-4e011d21bb01 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:40.340718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 817/1656] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:40.382305 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bd69e-11e0-4970-8651-1dbe0f4480f5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:40.385438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-666bd69e-11e0-4970-8651-1dbe0f4480f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:40.496582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bd69e-11e0-4970-8651-1dbe0f4480f5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:40.502010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:40.502790 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:40.502965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:40.503370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] PATH_INFO: `/v3/projects/472d4c74ede24fd691d42b3185e919b2` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:40.504998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=472d4c74ede24fd691d42b3185e919b2)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:40.506170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: WARNING py.warnings [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 12:20:40.506170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: warnings.warn(msg) Sep 21 12:20:40.506170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: Sep 21 12:20:40.509113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:40.533539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:40.629857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '472d4c74ede24fd691d42b3185e919b2', 'request_id': 'req-102fd84f-7683-4e76-8d02-a35b32e6b9fd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:40.662364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '472d4c74ede24fd691d42b3185e919b2', 'request_id': 'req-102fd84f-7683-4e76-8d02-a35b32e6b9fd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:40.679938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '472d4c74ede24fd691d42b3185e919b2', 'request_id': 'req-102fd84f-7683-4e76-8d02-a35b32e6b9fd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:40.714594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] The token cache is being invalidate because project 472d4c74ede24fd691d42b3185e919b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:20:40.714897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:40.717477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:40.721434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 840/1657] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:40 2022] DELETE /identity/v3/projects/472d4c74ede24fd691d42b3185e919b2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:40.725159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2604c2b-e60d-4a93-b09c-4e011d21bb01 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:40.744932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2604c2b-e60d-4a93-b09c-4e011d21bb01 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:40.754331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:40.882907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:40.883964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:40.884245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:40.884524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] PATH_INFO: `/v3/users/989594847219494db6b94ea95ca7ef22` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:40.887486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=989594847219494db6b94ea95ca7ef22)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:40.891496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:40.995066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '989594847219494db6b94ea95ca7ef22'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:40.996145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '989594847219494db6b94ea95ca7ef22'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:41.017143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '989594847219494db6b94ea95ca7ef22'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:41.018156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '989594847219494db6b94ea95ca7ef22'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:41.037771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '989594847219494db6b94ea95ca7ef22'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:41.089041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 818/1658] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:40 2022] DELETE /identity/v3/users/989594847219494db6b94ea95ca7ef22 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:42.053635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:42.060704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:42.093424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fd84f-7683-4e76-8d02-a35b32e6b9fd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:42.224215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:42.224215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:42.270971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:42.272453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:42.272742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:42.273024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] PATH_INFO: `/v3/projects/6b33841bf7d54fbea51c7d4ea55b7d50` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:42.279716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b33841bf7d54fbea51c7d4ea55b7d50)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:42.282404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:42.384234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa32593-629d-4673-87c4-c5cf630b0843 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:42.392592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b33841bf7d54fbea51c7d4ea55b7d50', 'request_id': 'req-957ea085-6422-4bde-ab12-74b1a15dcd4d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.405995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b33841bf7d54fbea51c7d4ea55b7d50', 'request_id': 'req-957ea085-6422-4bde-ab12-74b1a15dcd4d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.421825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b33841bf7d54fbea51c7d4ea55b7d50', 'request_id': 'req-957ea085-6422-4bde-ab12-74b1a15dcd4d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.442630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] The token cache is being invalidate because project 6b33841bf7d54fbea51c7d4ea55b7d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:20:42.443867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.445805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.451664 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 841/1659] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:42 2022] DELETE /identity/v3/projects/6b33841bf7d54fbea51c7d4ea55b7d50 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:42.453295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:42.458112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:42.458486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:42.571823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:42.574337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:42.575885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:42.576282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] PATH_INFO: `/v3/users/df8530247db44cf09912f1d2d15c1559` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:42.579160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df8530247db44cf09912f1d2d15c1559)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:42.583550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:42.643527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:42.727564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ea085-6422-4bde-ab12-74b1a15dcd4d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:42.756439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:42.759595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:42.760597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:42.765287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] PATH_INFO: `/v3/projects/25a6a74e37f54c63a74860f4119aa43f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:42.769122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a6a74e37f54c63a74860f4119aa43f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:42.774907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:42.779898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df8530247db44cf09912f1d2d15c1559'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.781010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df8530247db44cf09912f1d2d15c1559'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.798032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df8530247db44cf09912f1d2d15c1559'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.798764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df8530247db44cf09912f1d2d15c1559'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.815970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df8530247db44cf09912f1d2d15c1559'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.863007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 819/1660] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:42 2022] DELETE /identity/v3/users/df8530247db44cf09912f1d2d15c1559 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:42.864406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:42.872010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:42.904046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a6a74e37f54c63a74860f4119aa43f', 'request_id': 'req-a1df0c72-ae18-495b-9e23-721404d16adf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.930917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a6a74e37f54c63a74860f4119aa43f', 'request_id': 'req-a1df0c72-ae18-495b-9e23-721404d16adf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.971297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a6a74e37f54c63a74860f4119aa43f', 'request_id': 'req-a1df0c72-ae18-495b-9e23-721404d16adf'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:42.998559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] The token cache is being invalidate because project 25a6a74e37f54c63a74860f4119aa43f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:20:42.998812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:43.000615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a1df0c72-ae18-495b-9e23-721404d16adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:43.010899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 842/1661] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:42 2022] DELETE /identity/v3/projects/25a6a74e37f54c63a74860f4119aa43f => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:43.012596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44bc08-3b45-42ef-9dbd-ca84d3a02663 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:43.012891 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44bc08-3b45-42ef-9dbd-ca84d3a02663 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:43.013169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44bc08-3b45-42ef-9dbd-ca84d3a02663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:43.116308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eb44bc08-3b45-42ef-9dbd-ca84d3a02663 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:43.117558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eb44bc08-3b45-42ef-9dbd-ca84d3a02663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:43.133509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28d00f-c23d-4e1a-991a-9a517544f5b7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:43.371488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:43.372871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:43.374606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:43.375088 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] PATH_INFO: `/v3/users/b79b75bbd3304975b0a75ad8c5bda9a8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:43.382224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b79b75bbd3304975b0a75ad8c5bda9a8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:43.388194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:43.400659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 843/1662] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:43.404331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:43.406982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605600e-097d-4a2f-a032-2e984639fbcf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:43.407413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605600e-097d-4a2f-a032-2e984639fbcf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:43.408045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605600e-097d-4a2f-a032-2e984639fbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:43.490052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4605600e-097d-4a2f-a032-2e984639fbcf None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:43.495191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4605600e-097d-4a2f-a032-2e984639fbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:43.537410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 844/1663] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:43.538757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca890910-ec48-46f6-9767-e051a6af3d6f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:43.539040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca890910-ec48-46f6-9767-e051a6af3d6f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:43.539312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca890910-ec48-46f6-9767-e051a6af3d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:43.567690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b79b75bbd3304975b0a75ad8c5bda9a8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:43.568683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b79b75bbd3304975b0a75ad8c5bda9a8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:43.588756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b79b75bbd3304975b0a75ad8c5bda9a8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:43.593795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ca890910-ec48-46f6-9767-e051a6af3d6f None None] MFA Rules not processed for user `910d177b6f89421ca31e201fb82378f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:43.595666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ca890910-ec48-46f6-9767-e051a6af3d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:43.595849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b79b75bbd3304975b0a75ad8c5bda9a8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:43.616476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-848ac684-961c-407f-96f2-28a8f4ad877a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b79b75bbd3304975b0a75ad8c5bda9a8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:43.680143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 820/1664] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:42 2022] DELETE /identity/v3/users/b79b75bbd3304975b0a75ad8c5bda9a8 => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:43.697963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d84eec-9fe3-4670-8ed1-34c5e707a57f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:43.697963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d84eec-9fe3-4670-8ed1-34c5e707a57f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:43.697963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d84eec-9fe3-4670-8ed1-34c5e707a57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:43.740983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:43.758110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-00d84eec-9fe3-4670-8ed1-34c5e707a57f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:43.770427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-00d84eec-9fe3-4670-8ed1-34c5e707a57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:43.828818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 845/1665] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:43 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:43.836627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca890910-ec48-46f6-9767-e051a6af3d6f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:43.840750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ca890910-ec48-46f6-9767-e051a6af3d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:44.012047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca890910-ec48-46f6-9767-e051a6af3d6f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:44.031504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 821/1666] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:44.032598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d84eec-9fe3-4670-8ed1-34c5e707a57f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:44.056158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d84eec-9fe3-4670-8ed1-34c5e707a57f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:44.075401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bac766-f6d6-4660-a798-923f9d8a9f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:44.082226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bac766-f6d6-4660-a798-923f9d8a9f3f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:44.083045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bac766-f6d6-4660-a798-923f9d8a9f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:44.083045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bac766-f6d6-4660-a798-923f9d8a9f3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:44.089232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bac766-f6d6-4660-a798-923f9d8a9f3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:44.089571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bac766-f6d6-4660-a798-923f9d8a9f3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:44.091213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bac766-f6d6-4660-a798-923f9d8a9f3f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:44.133392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 822/1667] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:44.134894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bac766-f6d6-4660-a798-923f9d8a9f3f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:44.144158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b089329-dae2-471b-8d1b-8740e10f9c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:44.154026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b089329-dae2-471b-8d1b-8740e10f9c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:44.154487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b089329-dae2-471b-8d1b-8740e10f9c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:44.154921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b089329-dae2-471b-8d1b-8740e10f9c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:44.168718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2b089329-dae2-471b-8d1b-8740e10f9c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:44.169516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bac766-f6d6-4660-a798-923f9d8a9f3f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:44.192698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5428a-9dd3-4f06-ac8c-202a11b5603d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:44.193334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5428a-9dd3-4f06-ac8c-202a11b5603d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:44.193617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5428a-9dd3-4f06-ac8c-202a11b5603d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:44.193892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5428a-9dd3-4f06-ac8c-202a11b5603d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:44.194534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5428a-9dd3-4f06-ac8c-202a11b5603d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:44.197141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5428a-9dd3-4f06-ac8c-202a11b5603d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:44.202827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b089329-dae2-471b-8d1b-8740e10f9c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:44.205335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b089329-dae2-471b-8d1b-8740e10f9c3d service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:44.253378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 846/1668] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:44.260350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 823/1669] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:44 2022] POST /identity/v3/projects => generated 355 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:44.261858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b089329-dae2-471b-8d1b-8740e10f9c3d service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:44.289266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b089329-dae2-471b-8d1b-8740e10f9c3d service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:44.507071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8a023-ed2d-4466-9298-86aea056a7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:44.507684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8a023-ed2d-4466-9298-86aea056a7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:44.507987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8a023-ed2d-4466-9298-86aea056a7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:44.508266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8a023-ed2d-4466-9298-86aea056a7b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:44.508913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8a023-ed2d-4466-9298-86aea056a7b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:44.513917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8a023-ed2d-4466-9298-86aea056a7b4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:44.582121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 847/1670] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:44 2022] POST /identity/v3/users => generated 278 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:44.587389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5428a-9dd3-4f06-ac8c-202a11b5603d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:44.601333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5428a-9dd3-4f06-ac8c-202a11b5603d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:44.620020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16392a-8337-4786-bb35-ad59f116bc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:44.620667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16392a-8337-4786-bb35-ad59f116bc48 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:44.620944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16392a-8337-4786-bb35-ad59f116bc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:44.621216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16392a-8337-4786-bb35-ad59f116bc48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:44.622013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16392a-8337-4786-bb35-ad59f116bc48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:44.628591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16392a-8337-4786-bb35-ad59f116bc48 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:44.648515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 824/1671] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:44.652936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8a023-ed2d-4466-9298-86aea056a7b4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:44.667516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8a023-ed2d-4466-9298-86aea056a7b4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:44.679174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00f16c-2759-4e4e-b86d-bc704ed1fcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:44.679837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00f16c-2759-4e4e-b86d-bc704ed1fcfd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:44.680126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00f16c-2759-4e4e-b86d-bc704ed1fcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:44.680419 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00f16c-2759-4e4e-b86d-bc704ed1fcfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:44.681065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00f16c-2759-4e4e-b86d-bc704ed1fcfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:44.689740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00f16c-2759-4e4e-b86d-bc704ed1fcfd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:44.706239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 848/1672] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:44.710949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16392a-8337-4786-bb35-ad59f116bc48 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:44.726682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16392a-8337-4786-bb35-ad59f116bc48 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:44.740519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11362033-7630-4f39-9b28-9f8fcb4d7a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:44.745172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11362033-7630-4f39-9b28-9f8fcb4d7a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:44.745172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11362033-7630-4f39-9b28-9f8fcb4d7a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:44.745172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11362033-7630-4f39-9b28-9f8fcb4d7a59 admin admin] PATH_INFO: `/v3/projects/335f7a2759b74da186531f8293357695/users/03376fdb8a3b4cda9aaee54294fe3766/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:44.791026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11362033-7630-4f39-9b28-9f8fcb4d7a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=335f7a2759b74da186531f8293357695, user_id=03376fdb8a3b4cda9aaee54294fe3766, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:44.793715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11362033-7630-4f39-9b28-9f8fcb4d7a59 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:44.841618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 825/1673] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:44 2022] PUT /identity/v3/projects/335f7a2759b74da186531f8293357695/users/03376fdb8a3b4cda9aaee54294fe3766/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:44.852604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050d5a7-356f-48ad-9f90-508bf10bdcb5 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:44.852896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050d5a7-356f-48ad-9f90-508bf10bdcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:44.853183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050d5a7-356f-48ad-9f90-508bf10bdcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:44.939078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6050d5a7-356f-48ad-9f90-508bf10bdcb5 None None] MFA Rules not processed for user `03376fdb8a3b4cda9aaee54294fe3766`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:44.941804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6050d5a7-356f-48ad-9f90-508bf10bdcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:44.982219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:45.123899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 849/1674] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:45.130764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11362033-7630-4f39-9b28-9f8fcb4d7a59 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:45.157365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:45.269054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11362033-7630-4f39-9b28-9f8fcb4d7a59 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:45.284623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2710b-1ad4-4de5-babf-644c7218ef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:45.285185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2710b-1ad4-4de5-babf-644c7218ef4f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:45.285475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2710b-1ad4-4de5-babf-644c7218ef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:45.285752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2710b-1ad4-4de5-babf-644c7218ef4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:45.287832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2710b-1ad4-4de5-babf-644c7218ef4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:45.292427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2710b-1ad4-4de5-babf-644c7218ef4f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:45.395515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 826/1675] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:45 2022] POST /identity/v3/projects => generated 357 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:45.399977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050d5a7-356f-48ad-9f90-508bf10bdcb5 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:45.420196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050d5a7-356f-48ad-9f90-508bf10bdcb5 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:45.605057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f842f3-fbb8-4498-b2c3-545b38833e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:45.605994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f842f3-fbb8-4498-b2c3-545b38833e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:45.606471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f842f3-fbb8-4498-b2c3-545b38833e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:45.606879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f842f3-fbb8-4498-b2c3-545b38833e6f admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:45.610986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f842f3-fbb8-4498-b2c3-545b38833e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:45.613190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f842f3-fbb8-4498-b2c3-545b38833e6f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:45.666405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 850/1676] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:45 2022] POST /identity/v3/users => generated 285 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:45.670403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2710b-1ad4-4de5-babf-644c7218ef4f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:45.692803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2710b-1ad4-4de5-babf-644c7218ef4f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:45.700430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff99533-9990-4ba0-84de-6ad24f43a0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:45.702648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff99533-9990-4ba0-84de-6ad24f43a0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:45.704444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff99533-9990-4ba0-84de-6ad24f43a0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:45.704989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff99533-9990-4ba0-84de-6ad24f43a0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:45.706149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff99533-9990-4ba0-84de-6ad24f43a0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:45.708527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff99533-9990-4ba0-84de-6ad24f43a0ce admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:45.741151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 827/1677] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:20:45.746315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f842f3-fbb8-4498-b2c3-545b38833e6f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:45.778011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f842f3-fbb8-4498-b2c3-545b38833e6f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:45.791471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81165707-79d1-4410-843c-72a6e2e82ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:45.794504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81165707-79d1-4410-843c-72a6e2e82ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:45.795964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81165707-79d1-4410-843c-72a6e2e82ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:45.797976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81165707-79d1-4410-843c-72a6e2e82ac3 admin admin] PATH_INFO: `/v3/projects/73ed7794d48c4878957afe8b2bb168fa/users/7d4fc26327714fe2abce6fa16b837b11/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:45.872248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81165707-79d1-4410-843c-72a6e2e82ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ed7794d48c4878957afe8b2bb168fa, user_id=7d4fc26327714fe2abce6fa16b837b11, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:45.879122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81165707-79d1-4410-843c-72a6e2e82ac3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:45.918925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 851/1678] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:45 2022] PUT /identity/v3/projects/73ed7794d48c4878957afe8b2bb168fa/users/7d4fc26327714fe2abce6fa16b837b11/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:45.920464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff99533-9990-4ba0-84de-6ad24f43a0ce admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:45.935616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff99533-9990-4ba0-84de-6ad24f43a0ce admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:46.099511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989b475-d660-4bab-ae2f-92ea7f595039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:46.100031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989b475-d660-4bab-ae2f-92ea7f595039 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:46.100483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989b475-d660-4bab-ae2f-92ea7f595039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:46.100850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989b475-d660-4bab-ae2f-92ea7f595039 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:46.101512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989b475-d660-4bab-ae2f-92ea7f595039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:46.104985 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989b475-d660-4bab-ae2f-92ea7f595039 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:46.135814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 828/1679] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:20:46.137152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81165707-79d1-4410-843c-72a6e2e82ac3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:46.184490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81165707-79d1-4410-843c-72a6e2e82ac3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:46.200064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60fe6b-d586-4fc6-ba48-9bd5c6987898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:46.200770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60fe6b-d586-4fc6-ba48-9bd5c6987898 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:46.201855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60fe6b-d586-4fc6-ba48-9bd5c6987898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:46.202701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60fe6b-d586-4fc6-ba48-9bd5c6987898 admin admin] PATH_INFO: `/v3/projects/73ed7794d48c4878957afe8b2bb168fa/users/7d4fc26327714fe2abce6fa16b837b11/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:46.212956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60fe6b-d586-4fc6-ba48-9bd5c6987898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ed7794d48c4878957afe8b2bb168fa, user_id=7d4fc26327714fe2abce6fa16b837b11, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:46.221950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60fe6b-d586-4fc6-ba48-9bd5c6987898 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:46.262119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 852/1680] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:46 2022] PUT /identity/v3/projects/73ed7794d48c4878957afe8b2bb168fa/users/7d4fc26327714fe2abce6fa16b837b11/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:46.273173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334054bd-7bc9-4598-b420-c4f94bf64039 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:46.273845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334054bd-7bc9-4598-b420-c4f94bf64039 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:46.274324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334054bd-7bc9-4598-b420-c4f94bf64039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:46.346028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-334054bd-7bc9-4598-b420-c4f94bf64039 None None] MFA Rules not processed for user `7d4fc26327714fe2abce6fa16b837b11`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:46.347537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-334054bd-7bc9-4598-b420-c4f94bf64039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:46.362985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60fe6b-d586-4fc6-ba48-9bd5c6987898 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:46.365397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8d60fe6b-d586-4fc6-ba48-9bd5c6987898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:46.542685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60fe6b-d586-4fc6-ba48-9bd5c6987898 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:46.556152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:46.558763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:46.559642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:46.561279 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] PATH_INFO: `/v3/projects/e2aa4b291f4e4c8ba26e7c136b33b5b8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:46.565276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 829/1681] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:46.566117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2aa4b291f4e4c8ba26e7c136b33b5b8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:46.571718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:46.579598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334054bd-7bc9-4598-b420-c4f94bf64039 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:46.583004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-334054bd-7bc9-4598-b420-c4f94bf64039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:46.590334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:46.595624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:46.722055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2aa4b291f4e4c8ba26e7c136b33b5b8', 'request_id': 'req-c5790ae9-1be5-4430-a24c-942b2531b0ce'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:46.738777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2aa4b291f4e4c8ba26e7c136b33b5b8', 'request_id': 'req-c5790ae9-1be5-4430-a24c-942b2531b0ce'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:46.761368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334054bd-7bc9-4598-b420-c4f94bf64039 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:46.764176 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2aa4b291f4e4c8ba26e7c136b33b5b8', 'request_id': 'req-c5790ae9-1be5-4430-a24c-942b2531b0ce'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:46.800918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] The token cache is being invalidate because project e2aa4b291f4e4c8ba26e7c136b33b5b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:20:46.801268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:46.802127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5790ae9-1be5-4430-a24c-942b2531b0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:46.815820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 853/1682] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:46 2022] DELETE /identity/v3/projects/e2aa4b291f4e4c8ba26e7c136b33b5b8 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:46.817517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d64217-db73-49c3-a0c6-f827c8e99803 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:46.817805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d64217-db73-49c3-a0c6-f827c8e99803 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:46.818249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d64217-db73-49c3-a0c6-f827c8e99803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:46.889913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-64d64217-db73-49c3-a0c6-f827c8e99803 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:46.903950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64d64217-db73-49c3-a0c6-f827c8e99803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:46.918292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4863d2-5d02-41d2-9b7c-3076f409ba23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:46.919085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4863d2-5d02-41d2-9b7c-3076f409ba23 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:46.919668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4863d2-5d02-41d2-9b7c-3076f409ba23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:46.920212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4863d2-5d02-41d2-9b7c-3076f409ba23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:46.924368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8c4863d2-5d02-41d2-9b7c-3076f409ba23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:47.085671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 854/1683] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:47.085671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d64217-db73-49c3-a0c6-f827c8e99803 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:47.113888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64d64217-db73-49c3-a0c6-f827c8e99803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:47.145062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d64217-db73-49c3-a0c6-f827c8e99803 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:47.150788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:47.151529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:47.151934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:47.152343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] PATH_INFO: `/v3/projects/1ca32cb607f84222a9f547700f6c5fe4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:47.154202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ca32cb607f84222a9f547700f6c5fe4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:47.157812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:47.163797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4863d2-5d02-41d2-9b7c-3076f409ba23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:47.165369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4863d2-5d02-41d2-9b7c-3076f409ba23 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:47.209947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 830/1684] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:47.211925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be712-9e5d-406e-b84b-7f1346ffe76f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:47.212216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be712-9e5d-406e-b84b-7f1346ffe76f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:47.212496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195be712-9e5d-406e-b84b-7f1346ffe76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:47.269866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-195be712-9e5d-406e-b84b-7f1346ffe76f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:47.271228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-195be712-9e5d-406e-b84b-7f1346ffe76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:47.301466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 831/1685] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:47.306135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195be712-9e5d-406e-b84b-7f1346ffe76f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:47.334847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195be712-9e5d-406e-b84b-7f1346ffe76f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:47.372984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ca32cb607f84222a9f547700f6c5fe4', 'request_id': 'req-a3967102-7129-477b-b2b5-c4a992c1ed96'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:47.428564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ca32cb607f84222a9f547700f6c5fe4', 'request_id': 'req-a3967102-7129-477b-b2b5-c4a992c1ed96'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:47.449055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ca32cb607f84222a9f547700f6c5fe4', 'request_id': 'req-a3967102-7129-477b-b2b5-c4a992c1ed96'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:47.490269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] The token cache is being invalidate because project 1ca32cb607f84222a9f547700f6c5fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:20:47.490269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:47.491134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a3967102-7129-477b-b2b5-c4a992c1ed96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:47.492849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 855/1686] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:47 2022] DELETE /identity/v3/projects/1ca32cb607f84222a9f547700f6c5fe4 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:47.494118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd043c5-a2d1-4748-86d5-e9908d232625 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:47.494404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd043c5-a2d1-4748-86d5-e9908d232625 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:47.494687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd043c5-a2d1-4748-86d5-e9908d232625 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:47.496239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 856/1687] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:20:47.501918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40a007-388a-443a-ab45-e1eb40fa0b0c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:47.502241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40a007-388a-443a-ab45-e1eb40fa0b0c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:47.502475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40a007-388a-443a-ab45-e1eb40fa0b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:47.532775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:47.533638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:47.534249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:47.534530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:47.535413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:47.535890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:47.539120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:47.552985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fb40a007-388a-443a-ab45-e1eb40fa0b0c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:47.554292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb40a007-388a-443a-ab45-e1eb40fa0b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:47.559282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:47.574763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 832/1688] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:20:47.574763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:47.578586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:47.618579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 857/1689] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:47.625411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe009f7-7698-48dd-ae4c-919175d20479 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:47.625411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe009f7-7698-48dd-ae4c-919175d20479 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:47.625411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe009f7-7698-48dd-ae4c-919175d20479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:47.667623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7fe009f7-7698-48dd-ae4c-919175d20479 None None] MFA Rules not processed for user `eac29fd43f1f4bbd987950b90385200b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:47.672392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7fe009f7-7698-48dd-ae4c-919175d20479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:47.689463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:47.769367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26d94a-0fed-4354-b846-2480899ad8db admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:47.776519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196ce82-baa1-4e4f-b102-6e00c615548f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:47.777137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196ce82-baa1-4e4f-b102-6e00c615548f service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:47.777406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196ce82-baa1-4e4f-b102-6e00c615548f service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:47.777684 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196ce82-baa1-4e4f-b102-6e00c615548f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:47.783904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4196ce82-baa1-4e4f-b102-6e00c615548f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:47.807643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 858/1690] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:47.808957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe009f7-7698-48dd-ae4c-919175d20479 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:47.810499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7fe009f7-7698-48dd-ae4c-919175d20479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:47.827182 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe009f7-7698-48dd-ae4c-919175d20479 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:47.839455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eaf3a4-441a-4765-b0d4-367735e1f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:47.840546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaf3a4-441a-4765-b0d4-367735e1f9ee admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:47.843106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaf3a4-441a-4765-b0d4-367735e1f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:47.844725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaf3a4-441a-4765-b0d4-367735e1f9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:47.847470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eaf3a4-441a-4765-b0d4-367735e1f9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:47.851656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eaf3a4-441a-4765-b0d4-367735e1f9ee admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:47.943715 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 859/1691] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:47 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:47.949065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c373768-b41a-4534-a679-6e562335dbdf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:47.949811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c373768-b41a-4534-a679-6e562335dbdf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:47.950108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c373768-b41a-4534-a679-6e562335dbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:47.988393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196ce82-baa1-4e4f-b102-6e00c615548f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:47.991733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196ce82-baa1-4e4f-b102-6e00c615548f service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.014774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7c373768-b41a-4534-a679-6e562335dbdf None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:48.017321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7c373768-b41a-4534-a679-6e562335dbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:48.025212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 833/1692] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:48.032181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196ce82-baa1-4e4f-b102-6e00c615548f service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.069346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196ce82-baa1-4e4f-b102-6e00c615548f service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.207359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 860/1693] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:48.209666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c373768-b41a-4534-a679-6e562335dbdf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.214632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7c373768-b41a-4534-a679-6e562335dbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:48.228198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b775-7820-4fe5-a754-cef6f0a721b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:48.229050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b775-7820-4fe5-a754-cef6f0a721b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.230626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b775-7820-4fe5-a754-cef6f0a721b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.230626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076b775-7820-4fe5-a754-cef6f0a721b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.230626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076b775-7820-4fe5-a754-cef6f0a721b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:48.234561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076b775-7820-4fe5-a754-cef6f0a721b8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.307275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 834/1694] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:48 2022] POST /identity/v3/users => generated 284 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:48.308562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b775-7820-4fe5-a754-cef6f0a721b8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.321599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076b775-7820-4fe5-a754-cef6f0a721b8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.339509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef36719-3345-48c7-b9cd-2e90515e1885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:48.343025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef36719-3345-48c7-b9cd-2e90515e1885 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.343658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef36719-3345-48c7-b9cd-2e90515e1885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.343658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef36719-3345-48c7-b9cd-2e90515e1885 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.344522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef36719-3345-48c7-b9cd-2e90515e1885 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:48.344896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef36719-3345-48c7-b9cd-2e90515e1885 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:48.346631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef36719-3345-48c7-b9cd-2e90515e1885 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.376449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 835/1695] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:48.377759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef36719-3345-48c7-b9cd-2e90515e1885 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.395396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef36719-3345-48c7-b9cd-2e90515e1885 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.401807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7213f19-4e69-4481-b08b-2b6afef8a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:48.402609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7213f19-4e69-4481-b08b-2b6afef8a64e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.402847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7213f19-4e69-4481-b08b-2b6afef8a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.403187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7213f19-4e69-4481-b08b-2b6afef8a64e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.403987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7213f19-4e69-4481-b08b-2b6afef8a64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:48.406260 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7213f19-4e69-4481-b08b-2b6afef8a64e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.417849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c373768-b41a-4534-a679-6e562335dbdf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.430425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e4f81-0092-44be-8086-eb4c350a2eed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:48.431212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e4f81-0092-44be-8086-eb4c350a2eed service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.431657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e4f81-0092-44be-8086-eb4c350a2eed service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.431990 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e4f81-0092-44be-8086-eb4c350a2eed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.433873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 836/1696] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:48.436818 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7213f19-4e69-4481-b08b-2b6afef8a64e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.458570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e4f81-0092-44be-8086-eb4c350a2eed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:48.460213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e4f81-0092-44be-8086-eb4c350a2eed service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.472632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7213f19-4e69-4481-b08b-2b6afef8a64e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.492333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6056d7-cfcc-42ac-b371-2172842183f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:48.493237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6056d7-cfcc-42ac-b371-2172842183f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.493538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6056d7-cfcc-42ac-b371-2172842183f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.493818 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6056d7-cfcc-42ac-b371-2172842183f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.494473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6056d7-cfcc-42ac-b371-2172842183f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:48.499929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 861/1697] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:48.500911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6056d7-cfcc-42ac-b371-2172842183f0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.501011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e4f81-0092-44be-8086-eb4c350a2eed service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.535961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e4f81-0092-44be-8086-eb4c350a2eed service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.544874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d8511-df8e-4960-8e07-719417974107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:48.546185 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8511-df8e-4960-8e07-719417974107 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.546562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8511-df8e-4960-8e07-719417974107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.546740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d8511-df8e-4960-8e07-719417974107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.547333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:48.547422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d8511-df8e-4960-8e07-719417974107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:48.549064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d8511-df8e-4960-8e07-719417974107 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.573844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 837/1698] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:48 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:48.577952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6056d7-cfcc-42ac-b371-2172842183f0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.579825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 862/1699] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:48.585179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d8511-df8e-4960-8e07-719417974107 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.606832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:48.623104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d8511-df8e-4960-8e07-719417974107 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.721666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6056d7-cfcc-42ac-b371-2172842183f0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.736749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c997ee-e447-4c0d-85b3-88377f038b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:48.737402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c997ee-e447-4c0d-85b3-88377f038b0f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.737688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c997ee-e447-4c0d-85b3-88377f038b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.737962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c997ee-e447-4c0d-85b3-88377f038b0f admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.739000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c997ee-e447-4c0d-85b3-88377f038b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:48.740582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c997ee-e447-4c0d-85b3-88377f038b0f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.752931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c35df-4d8c-4bdb-b436-7b61bad115ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:48.753835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c35df-4d8c-4bdb-b436-7b61bad115ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.754013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c35df-4d8c-4bdb-b436-7b61bad115ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.754284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c35df-4d8c-4bdb-b436-7b61bad115ae admin admin] PATH_INFO: `/v3/projects/5f22fbc5daa94080bebad8757df9655e/users/5bc3315ec61549f5adeeadfe84392a66/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.786919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 838/1700] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:48 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:48.787943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c997ee-e447-4c0d-85b3-88377f038b0f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.789896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c35df-4d8c-4bdb-b436-7b61bad115ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f22fbc5daa94080bebad8757df9655e, user_id=5bc3315ec61549f5adeeadfe84392a66, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:48.792644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c35df-4d8c-4bdb-b436-7b61bad115ae admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.811020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c997ee-e447-4c0d-85b3-88377f038b0f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.817439 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e2bc3-4f37-45e8-9124-97e2b81f3ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:48.818069 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e2bc3-4f37-45e8-9124-97e2b81f3ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.818419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e2bc3-4f37-45e8-9124-97e2b81f3ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.819014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e2bc3-4f37-45e8-9124-97e2b81f3ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.819471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e2bc3-4f37-45e8-9124-97e2b81f3ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:48.821247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e2bc3-4f37-45e8-9124-97e2b81f3ae5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:48.831452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 863/1701] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:48 2022] PUT /identity/v3/projects/5f22fbc5daa94080bebad8757df9655e/users/5bc3315ec61549f5adeeadfe84392a66/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:48.838030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c06732-7214-46f3-ac51-868f1571c172 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:48.838310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c06732-7214-46f3-ac51-868f1571c172 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:48.838583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c06732-7214-46f3-ac51-868f1571c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:48.851795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 839/1702] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:20:48.853383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e2bc3-4f37-45e8-9124-97e2b81f3ae5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:48.870757 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e2bc3-4f37-45e8-9124-97e2b81f3ae5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:48.900017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-14c06732-7214-46f3-ac51-868f1571c172 None None] MFA Rules not processed for user `5bc3315ec61549f5adeeadfe84392a66`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:48.901258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-14c06732-7214-46f3-ac51-868f1571c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:49.001209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10d479-0f72-475d-a4e5-42e5cf422cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:49.002586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10d479-0f72-475d-a4e5-42e5cf422cca admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:49.003224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10d479-0f72-475d-a4e5-42e5cf422cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:49.004011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10d479-0f72-475d-a4e5-42e5cf422cca admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:49.006445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10d479-0f72-475d-a4e5-42e5cf422cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:49.011011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c10d479-0f72-475d-a4e5-42e5cf422cca admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:49.036298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 840/1703] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:20:49.037805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10d479-0f72-475d-a4e5-42e5cf422cca admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:49.060499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10d479-0f72-475d-a4e5-42e5cf422cca admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:49.072255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f45ea0-6ecc-4c29-8d01-0fdc0fe8b6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:49.075795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f45ea0-6ecc-4c29-8d01-0fdc0fe8b6b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:49.075795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f45ea0-6ecc-4c29-8d01-0fdc0fe8b6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:49.075795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f45ea0-6ecc-4c29-8d01-0fdc0fe8b6b3 admin admin] PATH_INFO: `/v3/projects/94cc1c11f48c46f9abddda990eaf64e8/users/21a25d78535c414d9080c6244fabeafd/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:49.076056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 864/1704] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:20:49.079858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c06732-7214-46f3-ac51-868f1571c172 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:49.104174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c06732-7214-46f3-ac51-868f1571c172 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:49.116707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f45ea0-6ecc-4c29-8d01-0fdc0fe8b6b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94cc1c11f48c46f9abddda990eaf64e8, user_id=21a25d78535c414d9080c6244fabeafd, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:49.119156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f7064-c53d-4ef6-b2fd-cf02fb3431cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:49.120753 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f7064-c53d-4ef6-b2fd-cf02fb3431cd admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:49.122508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f7064-c53d-4ef6-b2fd-cf02fb3431cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:49.122926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f7064-c53d-4ef6-b2fd-cf02fb3431cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:49.125879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f45ea0-6ecc-4c29-8d01-0fdc0fe8b6b3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:49.126025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f7064-c53d-4ef6-b2fd-cf02fb3431cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:49.129930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f7064-c53d-4ef6-b2fd-cf02fb3431cd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:49.158262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 841/1705] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:49 2022] PUT /identity/v3/projects/94cc1c11f48c46f9abddda990eaf64e8/users/21a25d78535c414d9080c6244fabeafd/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:49.168803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28cf06-73fe-4a1e-a045-dd330d29c382 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:49.168803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28cf06-73fe-4a1e-a045-dd330d29c382 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:49.168803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28cf06-73fe-4a1e-a045-dd330d29c382 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:49.168803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 842/1706] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:20:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:20:49.168803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28cf06-73fe-4a1e-a045-dd330d29c382 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:49.168803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1a28cf06-73fe-4a1e-a045-dd330d29c382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:49.206894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 865/1707] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:49 2022] POST /identity/v3/projects => generated 367 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:49.206894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b17d6-f612-42df-b7f6-0e93a6c190c4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:49.207506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b17d6-f612-42df-b7f6-0e93a6c190c4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:49.208670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b17d6-f612-42df-b7f6-0e93a6c190c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:49.285919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-591b17d6-f612-42df-b7f6-0e93a6c190c4 None None] MFA Rules not processed for user `21a25d78535c414d9080c6244fabeafd`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:49.287550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-591b17d6-f612-42df-b7f6-0e93a6c190c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:49.303914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:49.351634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28cf06-73fe-4a1e-a045-dd330d29c382 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:49.411345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:49.451515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:49.453607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:49.455173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:49.455561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] PATH_INFO: `/v3/users/3ff1bb4a99444649bacccded0db5adbc` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:49.461202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 866/1708] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:49.461378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff1bb4a99444649bacccded0db5adbc)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:49.461464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b17d6-f612-42df-b7f6-0e93a6c190c4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:49.461565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:49.488178 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b17d6-f612-42df-b7f6-0e93a6c190c4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:49.497979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16060bb3-40d2-4146-bef9-551bc76c8e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:49.499171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16060bb3-40d2-4146-bef9-551bc76c8e44 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:49.499465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16060bb3-40d2-4146-bef9-551bc76c8e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:49.499743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16060bb3-40d2-4146-bef9-551bc76c8e44 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:49.500817 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16060bb3-40d2-4146-bef9-551bc76c8e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:49.502803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16060bb3-40d2-4146-bef9-551bc76c8e44 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:49.564669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 867/1709] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:49 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:49.564669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3191f9-d661-451b-872a-c5affcbf1060 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:49.564669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3191f9-d661-451b-872a-c5affcbf1060 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:49.564669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3191f9-d661-451b-872a-c5affcbf1060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:49.565865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff1bb4a99444649bacccded0db5adbc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:49.569716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff1bb4a99444649bacccded0db5adbc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:49.583625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff1bb4a99444649bacccded0db5adbc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:49.584322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff1bb4a99444649bacccded0db5adbc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:49.603938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff1bb4a99444649bacccded0db5adbc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:49.620954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2f3191f9-d661-451b-872a-c5affcbf1060 None None] MFA Rules not processed for user `eac29fd43f1f4bbd987950b90385200b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:49.634383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f3191f9-d661-451b-872a-c5affcbf1060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:49.657441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 843/1710] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:49 2022] DELETE /identity/v3/users/3ff1bb4a99444649bacccded0db5adbc => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:49.660244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:49.662221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:49.830211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 868/1711] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:20:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:49.831117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3191f9-d661-451b-872a-c5affcbf1060 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:49.839642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f3191f9-d661-451b-872a-c5affcbf1060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:49.870713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3191f9-d661-451b-872a-c5affcbf1060 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:49.938339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07b805-c5d1-496d-917a-0d175430a903 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:49.967330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9a834-cbc6-4971-bacc-967ebc1266b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:49.968679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9a834-cbc6-4971-bacc-967ebc1266b0 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:49.968973 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9a834-cbc6-4971-bacc-967ebc1266b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:49.969393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9a834-cbc6-4971-bacc-967ebc1266b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:49.971932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-25c9a834-cbc6-4971-bacc-967ebc1266b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:50.009539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:50.010911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcec7d-504f-4a53-bd03-775b7aa82b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:50.011669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcec7d-504f-4a53-bd03-775b7aa82b3c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:50.011974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcec7d-504f-4a53-bd03-775b7aa82b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:50.012477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdcec7d-504f-4a53-bd03-775b7aa82b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:50.013259 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcec7d-504f-4a53-bd03-775b7aa82b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:50.020044 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdcec7d-504f-4a53-bd03-775b7aa82b3c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:50.066759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 869/1712] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:50.070005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcec7d-504f-4a53-bd03-775b7aa82b3c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:50.078850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dcdcec7d-504f-4a53-bd03-775b7aa82b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:50.159945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9a834-cbc6-4971-bacc-967ebc1266b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:50.161979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9a834-cbc6-4971-bacc-967ebc1266b0 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:50.176390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 844/1713] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:50.177786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9a834-cbc6-4971-bacc-967ebc1266b0 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:50.207235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9a834-cbc6-4971-bacc-967ebc1266b0 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:50.226015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7225d80-5664-449a-957e-18ebbf2ea259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:50.226844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7225d80-5664-449a-957e-18ebbf2ea259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:50.227392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7225d80-5664-449a-957e-18ebbf2ea259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:50.227823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7225d80-5664-449a-957e-18ebbf2ea259 admin admin] PATH_INFO: `/v3/projects/17b036ef79454dc1afd752c2e155bd18/users/35fb1abceb0c4459a2336f4c973e5a5d/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:50.281690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7225d80-5664-449a-957e-18ebbf2ea259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17b036ef79454dc1afd752c2e155bd18, user_id=35fb1abceb0c4459a2336f4c973e5a5d, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:50.285410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7225d80-5664-449a-957e-18ebbf2ea259 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:50.319798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 845/1714] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:50 2022] PUT /identity/v3/projects/17b036ef79454dc1afd752c2e155bd18/users/35fb1abceb0c4459a2336f4c973e5a5d/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:50.321342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deb3d0-f890-4586-9bd9-ca150bcaf653 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:50.324690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deb3d0-f890-4586-9bd9-ca150bcaf653 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:50.325163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12deb3d0-f890-4586-9bd9-ca150bcaf653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:50.334767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:50.353865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdcec7d-504f-4a53-bd03-775b7aa82b3c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:50.403150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-12deb3d0-f890-4586-9bd9-ca150bcaf653 None None] MFA Rules not processed for user `c950eb00b9474b29b86d24983e3b1583`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:50.404804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-12deb3d0-f890-4586-9bd9-ca150bcaf653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:50.491641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7f585-306b-4ad2-9b25-6021b9456835 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:50.491641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f585-306b-4ad2-9b25-6021b9456835 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:50.491641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f585-306b-4ad2-9b25-6021b9456835 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:50.491641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7f585-306b-4ad2-9b25-6021b9456835 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:50.500748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7f585-306b-4ad2-9b25-6021b9456835 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:50.502376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7f585-306b-4ad2-9b25-6021b9456835 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:50.514144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 870/1715] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:50.527243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7f585-306b-4ad2-9b25-6021b9456835 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:50.567951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 846/1716] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:50.569483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deb3d0-f890-4586-9bd9-ca150bcaf653 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:50.576150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-12deb3d0-f890-4586-9bd9-ca150bcaf653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:50.640000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7f585-306b-4ad2-9b25-6021b9456835 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:50.644733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033820a5-82ee-4a01-a85c-dff6c1eeb635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:50.645361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033820a5-82ee-4a01-a85c-dff6c1eeb635 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:50.645644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033820a5-82ee-4a01-a85c-dff6c1eeb635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:50.645919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033820a5-82ee-4a01-a85c-dff6c1eeb635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:50.646569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033820a5-82ee-4a01-a85c-dff6c1eeb635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:50.648378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033820a5-82ee-4a01-a85c-dff6c1eeb635 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:50.663391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 871/1717] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:50.664378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033820a5-82ee-4a01-a85c-dff6c1eeb635 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:50.666370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-033820a5-82ee-4a01-a85c-dff6c1eeb635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:50.744120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12deb3d0-f890-4586-9bd9-ca150bcaf653 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:50.755735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:50.756755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:50.757216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:50.758500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] PATH_INFO: `/v3/users/21a25d78535c414d9080c6244fabeafd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:50.760030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a25d78535c414d9080c6244fabeafd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:50.762953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:50.783404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033820a5-82ee-4a01-a85c-dff6c1eeb635 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:50.794360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:50.797139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970915fd-bb1f-45c3-b20b-8b1b0a7b445d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:50.800352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970915fd-bb1f-45c3-b20b-8b1b0a7b445d service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:50.800665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970915fd-bb1f-45c3-b20b-8b1b0a7b445d service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:50.800947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970915fd-bb1f-45c3-b20b-8b1b0a7b445d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:50.815707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970915fd-bb1f-45c3-b20b-8b1b0a7b445d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:50.817977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970915fd-bb1f-45c3-b20b-8b1b0a7b445d service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:50.833372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 872/1718] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:50.834531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970915fd-bb1f-45c3-b20b-8b1b0a7b445d service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:50.848392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970915fd-bb1f-45c3-b20b-8b1b0a7b445d service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:50.853940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a812dd-880a-411f-8299-6558864af723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:50.854922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a812dd-880a-411f-8299-6558864af723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:50.856015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a812dd-880a-411f-8299-6558864af723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:50.856524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a812dd-880a-411f-8299-6558864af723 admin admin] PATH_INFO: `/v3/projects/17b036ef79454dc1afd752c2e155bd18/users/35fb1abceb0c4459a2336f4c973e5a5d/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:50.860208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a812dd-880a-411f-8299-6558864af723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17b036ef79454dc1afd752c2e155bd18, user_id=35fb1abceb0c4459a2336f4c973e5a5d, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:50.864957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a812dd-880a-411f-8299-6558864af723 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:50.875087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a25d78535c414d9080c6244fabeafd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:50.876356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a25d78535c414d9080c6244fabeafd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:50.877602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:50.897879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 873/1719] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:50 2022] PUT /identity/v3/projects/17b036ef79454dc1afd752c2e155bd18/users/35fb1abceb0c4459a2336f4c973e5a5d/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:50.900905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ebf58-30b6-41ff-8391-4be5ef767903 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:50.901458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ebf58-30b6-41ff-8391-4be5ef767903 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:50.901915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ebf58-30b6-41ff-8391-4be5ef767903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:50.907145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a25d78535c414d9080c6244fabeafd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:50.908182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a25d78535c414d9080c6244fabeafd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:50.919363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-516fc590-1306-413d-82f5-0ec8985d3090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a25d78535c414d9080c6244fabeafd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:50.964172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c88ebf58-30b6-41ff-8391-4be5ef767903 None None] MFA Rules not processed for user `35fb1abceb0c4459a2336f4c973e5a5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:50.965590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c88ebf58-30b6-41ff-8391-4be5ef767903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:51.001314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 847/1720] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:50 2022] DELETE /identity/v3/users/21a25d78535c414d9080c6244fabeafd => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:51.007346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16232609-28ed-431e-b756-d4c99c8ba163 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:51.010203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16232609-28ed-431e-b756-d4c99c8ba163 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:51.010709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16232609-28ed-431e-b756-d4c99c8ba163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:51.093679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-16232609-28ed-431e-b756-d4c99c8ba163 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:51.094915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-16232609-28ed-431e-b756-d4c99c8ba163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:51.152986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 874/1721] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:51.156780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ebf58-30b6-41ff-8391-4be5ef767903 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:51.160326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c88ebf58-30b6-41ff-8391-4be5ef767903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:51.324464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 848/1722] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:51.325369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16232609-28ed-431e-b756-d4c99c8ba163 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:51.330640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-16232609-28ed-431e-b756-d4c99c8ba163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:51.371982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ebf58-30b6-41ff-8391-4be5ef767903 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:51.381856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:51.382610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:51.382899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:51.383175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] PATH_INFO: `/v3/users/5f1ea4662764421f8655851d1a937f57` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:51.384792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f1ea4662764421f8655851d1a937f57)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:51.387089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:51.517132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16232609-28ed-431e-b756-d4c99c8ba163 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:51.521828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f1ea4662764421f8655851d1a937f57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:51.526995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9a3aa-52cc-44a2-841a-beb6a1ea9537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:51.529485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9a3aa-52cc-44a2-841a-beb6a1ea9537 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:51.529898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9a3aa-52cc-44a2-841a-beb6a1ea9537 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:51.532370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9a3aa-52cc-44a2-841a-beb6a1ea9537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:51.538556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f1ea4662764421f8655851d1a937f57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:51.539410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f1ea4662764421f8655851d1a937f57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:51.540591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f1ea4662764421f8655851d1a937f57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:51.565472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f1ea4662764421f8655851d1a937f57'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:51.634652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 875/1723] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:51 2022] DELETE /identity/v3/users/5f1ea4662764421f8655851d1a937f57 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:51.637466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:51.640335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:51.664876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:51.675531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:51.713825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9a3aa-52cc-44a2-841a-beb6a1ea9537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:51.717415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9a3aa-52cc-44a2-841a-beb6a1ea9537 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:51.744720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 849/1724] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:51.745406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9a3aa-52cc-44a2-841a-beb6a1ea9537 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:51.764630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2bb9a3aa-52cc-44a2-841a-beb6a1ea9537 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:51.874055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a2ad9-0e96-411e-b6bc-5b4ae23d5dbe admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:51.883485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:51.885574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:51.885729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:51.887333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] PATH_INFO: `/v3/projects/bfe885783173412289ca18fb7cdc6bed` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:51.889922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe885783173412289ca18fb7cdc6bed)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:51.909426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:51.919325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9a3aa-52cc-44a2-841a-beb6a1ea9537 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:52.000704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:52.002193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:52.002757 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:52.002757 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:52.006552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:52.013679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe885783173412289ca18fb7cdc6bed', 'request_id': 'req-80b171fc-7bb9-41e7-8b4a-13748bb18c78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.025142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe885783173412289ca18fb7cdc6bed', 'request_id': 'req-80b171fc-7bb9-41e7-8b4a-13748bb18c78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.037955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe885783173412289ca18fb7cdc6bed', 'request_id': 'req-80b171fc-7bb9-41e7-8b4a-13748bb18c78'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.042282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:52.048803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:52.057026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] The token cache is being invalidate because project bfe885783173412289ca18fb7cdc6bed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:20:52.057385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.059059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-80b171fc-7bb9-41e7-8b4a-13748bb18c78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.061352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 876/1725] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:51 2022] DELETE /identity/v3/projects/bfe885783173412289ca18fb7cdc6bed => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:52.064311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504af7-cd1b-46f8-8efc-dd8a98f20007 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:52.064311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504af7-cd1b-46f8-8efc-dd8a98f20007 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:52.064311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67504af7-cd1b-46f8-8efc-dd8a98f20007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:52.069332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 850/1726] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:52.070221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:52.072633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:52.120982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-67504af7-cd1b-46f8-8efc-dd8a98f20007 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:52.124703 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-67504af7-cd1b-46f8-8efc-dd8a98f20007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:52.175623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79a0a0-6b37-453e-a4d8-9424fd7758f5 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:52.280766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:52.282481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:52.282937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:52.283663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:52.286855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:52.307669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:52.313064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 877/1727] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:52.320309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504af7-cd1b-46f8-8efc-dd8a98f20007 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:52.322209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-67504af7-cd1b-46f8-8efc-dd8a98f20007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:52.339114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67504af7-cd1b-46f8-8efc-dd8a98f20007 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:52.362435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:52.364288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:52.364929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:52.365159 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] PATH_INFO: `/v3/users/910d177b6f89421ca31e201fb82378f0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:52.366428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910d177b6f89421ca31e201fb82378f0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:52.368983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:52.386828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:52.388374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:52.402020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 851/1728] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:52.402977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:52.408101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:52.421898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:52.454191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910d177b6f89421ca31e201fb82378f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.473771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910d177b6f89421ca31e201fb82378f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.474509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910d177b6f89421ca31e201fb82378f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.476293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910d177b6f89421ca31e201fb82378f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.489647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910d177b6f89421ca31e201fb82378f0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:52.538572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 878/1729] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:52 2022] DELETE /identity/v3/users/910d177b6f89421ca31e201fb82378f0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:52.548536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741678c-20d5-47c5-bf8a-2b7c96e99c35 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:52.729081 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac814d7-53b8-4a86-8f8d-677a2957fc7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:52.729919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac814d7-53b8-4a86-8f8d-677a2957fc7e service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:52.730308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac814d7-53b8-4a86-8f8d-677a2957fc7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:52.730982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac814d7-53b8-4a86-8f8d-677a2957fc7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:52.734870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dac814d7-53b8-4a86-8f8d-677a2957fc7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:52.878163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:52.879711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:52.906568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac814d7-53b8-4a86-8f8d-677a2957fc7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:52.906568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac814d7-53b8-4a86-8f8d-677a2957fc7e service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:52.906934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1c938-69e8-4b79-b826-7e6e301579c4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:52.915175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:52.917823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:52.918225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:52.919636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 852/1730] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:52.920016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] PATH_INFO: `/v3/projects/94cc1c11f48c46f9abddda990eaf64e8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:52.924333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=94cc1c11f48c46f9abddda990eaf64e8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:52.926384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:53.034088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94cc1c11f48c46f9abddda990eaf64e8', 'request_id': 'req-34b2da8b-d593-4c53-ab1f-d15054760cfc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.051008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94cc1c11f48c46f9abddda990eaf64e8', 'request_id': 'req-34b2da8b-d593-4c53-ab1f-d15054760cfc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.070759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94cc1c11f48c46f9abddda990eaf64e8', 'request_id': 'req-34b2da8b-d593-4c53-ab1f-d15054760cfc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.090032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] The token cache is being invalidate because project 94cc1c11f48c46f9abddda990eaf64e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:20:53.090418 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.091060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.093839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 879/1731] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:52 2022] DELETE /identity/v3/projects/94cc1c11f48c46f9abddda990eaf64e8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:53.099373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39a296-5855-4c90-a48a-379867617ed1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:53.100384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39a296-5855-4c90-a48a-379867617ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:53.100885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39a296-5855-4c90-a48a-379867617ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:53.145297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:53.147003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:53.154116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cf39a296-5855-4c90-a48a-379867617ed1 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:53.156916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cf39a296-5855-4c90-a48a-379867617ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:53.206173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:53.319021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2da8b-d593-4c53-ab1f-d15054760cfc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:53.327142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 853/1732] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:53.332409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:53.333497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:53.333989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:53.334366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] PATH_INFO: `/v3/projects/aedd846e34fc471685daeec69f45b6cb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:53.335275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e73552-0f88-469d-a7be-ec42302e3062 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:53.335712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e73552-0f88-469d-a7be-ec42302e3062 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:53.336196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e73552-0f88-469d-a7be-ec42302e3062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:53.336815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aedd846e34fc471685daeec69f45b6cb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:53.342307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:53.363160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d4e73552-0f88-469d-a7be-ec42302e3062 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:53.366770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d4e73552-0f88-469d-a7be-ec42302e3062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:53.392896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 854/1733] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:53.393902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e73552-0f88-469d-a7be-ec42302e3062 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:53.408680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e73552-0f88-469d-a7be-ec42302e3062 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:53.416139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9df486-8a01-41aa-83b6-81a02f1a15d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:53.417285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9df486-8a01-41aa-83b6-81a02f1a15d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:53.417667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9df486-8a01-41aa-83b6-81a02f1a15d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:53.418042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9df486-8a01-41aa-83b6-81a02f1a15d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:53.418852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9df486-8a01-41aa-83b6-81a02f1a15d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:53.419303 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9df486-8a01-41aa-83b6-81a02f1a15d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:53.422018 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9df486-8a01-41aa-83b6-81a02f1a15d8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:53.429611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:53.453728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aedd846e34fc471685daeec69f45b6cb', 'request_id': 'req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.460102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 855/1734] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:53.462877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9df486-8a01-41aa-83b6-81a02f1a15d8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:53.476696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aedd846e34fc471685daeec69f45b6cb', 'request_id': 'req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.487466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9df486-8a01-41aa-83b6-81a02f1a15d8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:53.496211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aedd846e34fc471685daeec69f45b6cb', 'request_id': 'req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.527605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] The token cache is being invalidate because project aedd846e34fc471685daeec69f45b6cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:20:53.529351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.532793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-826b6da3-36a9-4cfe-8705-06aaa3fc5d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:53.537694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 880/1735] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:53 2022] DELETE /identity/v3/projects/aedd846e34fc471685daeec69f45b6cb => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:53.543968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd7ead-5f60-4f66-9afb-e039df9788dc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:53.544481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd7ead-5f60-4f66-9afb-e039df9788dc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:53.545027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd7ead-5f60-4f66-9afb-e039df9788dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:53.664479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-88cd7ead-5f60-4f66-9afb-e039df9788dc None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:53.665749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-88cd7ead-5f60-4f66-9afb-e039df9788dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:53.743837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9829d81-9546-43d0-b0c5-5be60d6d0348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:53.745389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829d81-9546-43d0-b0c5-5be60d6d0348 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:53.747969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829d81-9546-43d0-b0c5-5be60d6d0348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:53.749194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829d81-9546-43d0-b0c5-5be60d6d0348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:53.761765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9829d81-9546-43d0-b0c5-5be60d6d0348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:53.763962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9829d81-9546-43d0-b0c5-5be60d6d0348 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:53.832011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 881/1736] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:20:53.833899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cba1d-8900-4c0d-85d4-e42de2b31043 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:53.834222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cba1d-8900-4c0d-85d4-e42de2b31043 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:53.834572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cba1d-8900-4c0d-85d4-e42de2b31043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:53.856078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 856/1737] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:53 2022] POST /identity/v3/projects => generated 369 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:53.858139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9829d81-9546-43d0-b0c5-5be60d6d0348 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:53.874978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b9829d81-9546-43d0-b0c5-5be60d6d0348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:53.917731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d42cba1d-8900-4c0d-85d4-e42de2b31043 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:53.918997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d42cba1d-8900-4c0d-85d4-e42de2b31043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:54.057439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 882/1738] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:54.065737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cba1d-8900-4c0d-85d4-e42de2b31043 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.088757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cba1d-8900-4c0d-85d4-e42de2b31043 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.097104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b128dc-baee-4cba-a207-1deef3cccb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.097944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b128dc-baee-4cba-a207-1deef3cccb69 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.098332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b128dc-baee-4cba-a207-1deef3cccb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.098724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b128dc-baee-4cba-a207-1deef3cccb69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.100724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b128dc-baee-4cba-a207-1deef3cccb69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:54.101186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b128dc-baee-4cba-a207-1deef3cccb69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:54.103181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b128dc-baee-4cba-a207-1deef3cccb69 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:54.119736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 883/1739] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:54.124151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b128dc-baee-4cba-a207-1deef3cccb69 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.144847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b128dc-baee-4cba-a207-1deef3cccb69 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.150599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb53fd-2c86-4b8d-8bf4-f9d6de0dce9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.151338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb53fd-2c86-4b8d-8bf4-f9d6de0dce9d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.151745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb53fd-2c86-4b8d-8bf4-f9d6de0dce9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.155777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb53fd-2c86-4b8d-8bf4-f9d6de0dce9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.156862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb53fd-2c86-4b8d-8bf4-f9d6de0dce9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:54.161764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb53fd-2c86-4b8d-8bf4-f9d6de0dce9d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:54.161992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9829d81-9546-43d0-b0c5-5be60d6d0348 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.178071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae49142-6a71-43cd-bc06-5411622611f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.179064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae49142-6a71-43cd-bc06-5411622611f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.179528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae49142-6a71-43cd-bc06-5411622611f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.180171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae49142-6a71-43cd-bc06-5411622611f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.180849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae49142-6a71-43cd-bc06-5411622611f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:54.183120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae49142-6a71-43cd-bc06-5411622611f7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:54.240680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 884/1740] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:54 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:20:54.255045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 857/1741] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:53 2022] POST /identity/v3/users => generated 285 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:54.255248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb53fd-2c86-4b8d-8bf4-f9d6de0dce9d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.257295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae49142-6a71-43cd-bc06-5411622611f7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.291868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae49142-6a71-43cd-bc06-5411622611f7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.315329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:54.325857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:54.445450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb53fd-2c86-4b8d-8bf4-f9d6de0dce9d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.472316 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee2a5b-6aa3-47e3-b959-a761e342bc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.479619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee2a5b-6aa3-47e3-b959-a761e342bc6f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.479963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee2a5b-6aa3-47e3-b959-a761e342bc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.480245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee2a5b-6aa3-47e3-b959-a761e342bc6f admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.482257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee2a5b-6aa3-47e3-b959-a761e342bc6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:54.484568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee2a5b-6aa3-47e3-b959-a761e342bc6f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:54.506428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52a059-1b68-4a1f-8d14-f3d365a8f3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.507603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52a059-1b68-4a1f-8d14-f3d365a8f3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.507603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52a059-1b68-4a1f-8d14-f3d365a8f3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.507828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52a059-1b68-4a1f-8d14-f3d365a8f3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.508550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52a059-1b68-4a1f-8d14-f3d365a8f3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:54.518261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52a059-1b68-4a1f-8d14-f3d365a8f3a4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:54.535278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 858/1742] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:54.536262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52a059-1b68-4a1f-8d14-f3d365a8f3a4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.542648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 885/1743] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:54 2022] POST /identity/v3/users => generated 287 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:54.547495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee2a5b-6aa3-47e3-b959-a761e342bc6f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.553093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52a059-1b68-4a1f-8d14-f3d365a8f3a4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.566803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcce3df-3dc7-4dd7-ba7e-377f2dd568e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.567789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcce3df-3dc7-4dd7-ba7e-377f2dd568e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.568299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcce3df-3dc7-4dd7-ba7e-377f2dd568e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.568907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcce3df-3dc7-4dd7-ba7e-377f2dd568e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.569916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcce3df-3dc7-4dd7-ba7e-377f2dd568e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:54.572494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcce3df-3dc7-4dd7-ba7e-377f2dd568e6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:54.573101 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee2a5b-6aa3-47e3-b959-a761e342bc6f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.583946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0655e-2d0c-4b5a-ad49-2d5822d1fa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.585873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0655e-2d0c-4b5a-ad49-2d5822d1fa39 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.586159 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0655e-2d0c-4b5a-ad49-2d5822d1fa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.588458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0655e-2d0c-4b5a-ad49-2d5822d1fa39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.590095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0655e-2d0c-4b5a-ad49-2d5822d1fa39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:54.595533 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0655e-2d0c-4b5a-ad49-2d5822d1fa39 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:54.608606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 859/1744] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:54.609741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcce3df-3dc7-4dd7-ba7e-377f2dd568e6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.631179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcce3df-3dc7-4dd7-ba7e-377f2dd568e6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.634994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 886/1745] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:54.638345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0655e-2d0c-4b5a-ad49-2d5822d1fa39 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.652064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95747cb5-1d25-417b-80fc-fd2f7f6da664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.652677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95747cb5-1d25-417b-80fc-fd2f7f6da664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.653628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95747cb5-1d25-417b-80fc-fd2f7f6da664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.653628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95747cb5-1d25-417b-80fc-fd2f7f6da664 admin admin] PATH_INFO: `/v3/projects/ba887b47f56c472997515ee30031246e/users/f7a2802c5a42420c8d959be3bcc88c07/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.667933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0655e-2d0c-4b5a-ad49-2d5822d1fa39 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.691864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4147c-b745-4bcd-ae82-bda1723d1b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.693426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95747cb5-1d25-417b-80fc-fd2f7f6da664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba887b47f56c472997515ee30031246e, user_id=f7a2802c5a42420c8d959be3bcc88c07, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:54.693680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4147c-b745-4bcd-ae82-bda1723d1b8b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.694535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4147c-b745-4bcd-ae82-bda1723d1b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.695635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4147c-b745-4bcd-ae82-bda1723d1b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.696656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95747cb5-1d25-417b-80fc-fd2f7f6da664 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:54.697486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4147c-b745-4bcd-ae82-bda1723d1b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:54.703798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4147c-b745-4bcd-ae82-bda1723d1b8b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:54.739258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 860/1746] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:54 2022] PUT /identity/v3/projects/ba887b47f56c472997515ee30031246e/users/f7a2802c5a42420c8d959be3bcc88c07/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:54.743131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 887/1747] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:54.745930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17642fd-cf98-45ab-ae4c-16bc9d63ed8e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.745930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17642fd-cf98-45ab-ae4c-16bc9d63ed8e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.745930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17642fd-cf98-45ab-ae4c-16bc9d63ed8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:54.746211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4147c-b745-4bcd-ae82-bda1723d1b8b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.787228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4147c-b745-4bcd-ae82-bda1723d1b8b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:54.802304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b17642fd-cf98-45ab-ae4c-16bc9d63ed8e None None] MFA Rules not processed for user `f7a2802c5a42420c8d959be3bcc88c07`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:54.803727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b17642fd-cf98-45ab-ae4c-16bc9d63ed8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:54.953431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 861/1748] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:54.955011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17642fd-cf98-45ab-ae4c-16bc9d63ed8e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:54.979923 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449183c7-fb5b-4b21-a70f-e8fba3bd228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:54.980731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449183c7-fb5b-4b21-a70f-e8fba3bd228b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:54.980932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449183c7-fb5b-4b21-a70f-e8fba3bd228b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:54.981286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449183c7-fb5b-4b21-a70f-e8fba3bd228b admin admin] PATH_INFO: `/v3/projects/01322137ba0b464aad5a1b1d9499e8ed/users/1d73d5c579d94f428228e3dd4f231118/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:55.041638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449183c7-fb5b-4b21-a70f-e8fba3bd228b admin admin] RBAC: Authorizing `identity:create_grant(project_id=01322137ba0b464aad5a1b1d9499e8ed, user_id=1d73d5c579d94f428228e3dd4f231118, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:55.044374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449183c7-fb5b-4b21-a70f-e8fba3bd228b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:55.096138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 888/1749] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:54 2022] PUT /identity/v3/projects/01322137ba0b464aad5a1b1d9499e8ed/users/1d73d5c579d94f428228e3dd4f231118/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:55.097879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721068-ca33-4e7f-b416-ccf865308734 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:55.098187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721068-ca33-4e7f-b416-ccf865308734 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:55.098452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721068-ca33-4e7f-b416-ccf865308734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:55.144121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17642fd-cf98-45ab-ae4c-16bc9d63ed8e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:55.178129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a4721068-ca33-4e7f-b416-ccf865308734 None None] MFA Rules not processed for user `1d73d5c579d94f428228e3dd4f231118`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:55.179181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:55.179642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a4721068-ca33-4e7f-b416-ccf865308734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:55.372075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 889/1750] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:20:55.378859 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721068-ca33-4e7f-b416-ccf865308734 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:55.393055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e3cb-8673-429b-8833-8a3def2f2445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:55.399366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e3cb-8673-429b-8833-8a3def2f2445 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:55.400233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e3cb-8673-429b-8833-8a3def2f2445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:55.400525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e3cb-8673-429b-8833-8a3def2f2445 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:55.401373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4e3cb-8673-429b-8833-8a3def2f2445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:55.402157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721068-ca33-4e7f-b416-ccf865308734 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:55.406803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4e3cb-8673-429b-8833-8a3def2f2445 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:55.417476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350f1ec-6e23-449b-9932-181d81397214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:55.418200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350f1ec-6e23-449b-9932-181d81397214 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:55.418498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350f1ec-6e23-449b-9932-181d81397214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:55.418959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350f1ec-6e23-449b-9932-181d81397214 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:55.420693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350f1ec-6e23-449b-9932-181d81397214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:55.424341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350f1ec-6e23-449b-9932-181d81397214 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:55.445342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:55.452316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 862/1751] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:54 2022] POST /identity/v3/projects => generated 369 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:55.459024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e3cb-8673-429b-8833-8a3def2f2445 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:55.500567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 890/1752] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:55 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:55.505997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350f1ec-6e23-449b-9932-181d81397214 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:55.625206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e3cb-8673-429b-8833-8a3def2f2445 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:55.691607 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350f1ec-6e23-449b-9932-181d81397214 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:55.706279 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b61d3-9aac-4054-b422-914812850dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:55.706914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b61d3-9aac-4054-b422-914812850dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:55.707187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b61d3-9aac-4054-b422-914812850dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:55.707451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b61d3-9aac-4054-b422-914812850dd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:55.711543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b61d3-9aac-4054-b422-914812850dd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:55.713776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b61d3-9aac-4054-b422-914812850dd8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:55.761286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 891/1753] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:55 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:55.761286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b61d3-9aac-4054-b422-914812850dd8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:55.767347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-687b61d3-9aac-4054-b422-914812850dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:55.768110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbabc8-7eb8-4469-82bf-2e0a81f68d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:55.769041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbabc8-7eb8-4469-82bf-2e0a81f68d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:55.769475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbabc8-7eb8-4469-82bf-2e0a81f68d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:55.769897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbabc8-7eb8-4469-82bf-2e0a81f68d53 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:55.770838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbabc8-7eb8-4469-82bf-2e0a81f68d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:55.773515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbabc8-7eb8-4469-82bf-2e0a81f68d53 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:55.794554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b61d3-9aac-4054-b422-914812850dd8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:55.812897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:55.813560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:55.813839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:55.814118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] PATH_INFO: `/v3/projects/cf8d461fc9c14568b687b373990b17c8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:55.815337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf8d461fc9c14568b687b373990b17c8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:55.817899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:55.835640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 863/1754] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:55 2022] POST /identity/v3/users => generated 291 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:55.839791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d4d54-2a25-4baf-9811-3898f32370e7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:55.840832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d4d54-2a25-4baf-9811-3898f32370e7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:55.841847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d4d54-2a25-4baf-9811-3898f32370e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:55.919536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf8d461fc9c14568b687b373990b17c8', 'request_id': 'req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:55.924178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-891d4d54-2a25-4baf-9811-3898f32370e7 None None] MFA Rules not processed for user `2ab0433dd3864867b03c5dd6407aa388`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:55.925542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-891d4d54-2a25-4baf-9811-3898f32370e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:55.932007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf8d461fc9c14568b687b373990b17c8', 'request_id': 'req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:55.949018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf8d461fc9c14568b687b373990b17c8', 'request_id': 'req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:55.965328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:55.971631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] The token cache is being invalidate because project cf8d461fc9c14568b687b373990b17c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:20:55.972151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:55.975578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:55.979743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 892/1755] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:55 2022] DELETE /identity/v3/projects/cf8d461fc9c14568b687b373990b17c8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:55.980061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:55.985084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:56.035261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 864/1756] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:56.036300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d4d54-2a25-4baf-9811-3898f32370e7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:56.040980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-891d4d54-2a25-4baf-9811-3898f32370e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:56.074114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:56.087325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f5174-c009-431f-8c13-eb6d0ab4a45e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:56.101237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e714a77-f460-47d0-b0c8-32f94b75911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:56.107823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e714a77-f460-47d0-b0c8-32f94b75911f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.108304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e714a77-f460-47d0-b0c8-32f94b75911f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.108733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e714a77-f460-47d0-b0c8-32f94b75911f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.110362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e714a77-f460-47d0-b0c8-32f94b75911f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:56.113289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e714a77-f460-47d0-b0c8-32f94b75911f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:56.139568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 893/1757] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:56.139568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a62ac-c7da-49e5-aac4-1b4600998595 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.139568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a62ac-c7da-49e5-aac4-1b4600998595 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.139568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a62ac-c7da-49e5-aac4-1b4600998595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.170982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7b3a62ac-c7da-49e5-aac4-1b4600998595 None None] MFA Rules not processed for user `c950eb00b9474b29b86d24983e3b1583`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:56.172355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7b3a62ac-c7da-49e5-aac4-1b4600998595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:56.223528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d4d54-2a25-4baf-9811-3898f32370e7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:56.229693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac384fda-e14e-402e-8928-31b4144158b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:56.230643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac384fda-e14e-402e-8928-31b4144158b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.230926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac384fda-e14e-402e-8928-31b4144158b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.231976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac384fda-e14e-402e-8928-31b4144158b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.233422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac384fda-e14e-402e-8928-31b4144158b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:56.235302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac384fda-e14e-402e-8928-31b4144158b3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:56.247463 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 865/1758] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:56.255134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0ee47-6609-4cba-8165-e2e96bbecd57 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.255426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0ee47-6609-4cba-8165-e2e96bbecd57 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.255711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0ee47-6609-4cba-8165-e2e96bbecd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.309524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 894/1759] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:56.317026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bfb0ee47-6609-4cba-8165-e2e96bbecd57 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:56.319476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173327c-dff5-4bd6-88df-a1b4bb8a3dce None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.320090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bfb0ee47-6609-4cba-8165-e2e96bbecd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:56.320471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173327c-dff5-4bd6-88df-a1b4bb8a3dce None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.322325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173327c-dff5-4bd6-88df-a1b4bb8a3dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.339611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 866/1760] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:56.340761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0ee47-6609-4cba-8165-e2e96bbecd57 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:56.360529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0ee47-6609-4cba-8165-e2e96bbecd57 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:56.374442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4173327c-dff5-4bd6-88df-a1b4bb8a3dce None None] MFA Rules not processed for user `2ab0433dd3864867b03c5dd6407aa388`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:56.375864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4173327c-dff5-4bd6-88df-a1b4bb8a3dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:56.376515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f16e-069f-45b7-b0d1-c5ea402b3e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:56.378806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f16e-069f-45b7-b0d1-c5ea402b3e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.379392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f16e-069f-45b7-b0d1-c5ea402b3e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.382364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f16e-069f-45b7-b0d1-c5ea402b3e88 admin admin] PATH_INFO: `/v3/projects/f28391e42e2a4fe8b216495e3d6c68b7/users/e471de98af6545a3ba2f804be569cfc7/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.394416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 895/1761] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:56.395577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173327c-dff5-4bd6-88df-a1b4bb8a3dce None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:56.419891 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173327c-dff5-4bd6-88df-a1b4bb8a3dce None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:56.429371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33ea5c-c6ad-473f-aab0-c8fd6a130b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:56.431455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33ea5c-c6ad-473f-aab0-c8fd6a130b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.432791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33ea5c-c6ad-473f-aab0-c8fd6a130b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.434135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33ea5c-c6ad-473f-aab0-c8fd6a130b41 admin admin] PATH_INFO: `/v3/projects/396ec2cf84a34efe9cc84b30614db294/users/7cf8d956372b46fe960a81dc43633881/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.455456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4f16e-069f-45b7-b0d1-c5ea402b3e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28391e42e2a4fe8b216495e3d6c68b7, user_id=e471de98af6545a3ba2f804be569cfc7, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:56.458660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4f16e-069f-45b7-b0d1-c5ea402b3e88 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:56.476484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33ea5c-c6ad-473f-aab0-c8fd6a130b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=396ec2cf84a34efe9cc84b30614db294, user_id=7cf8d956372b46fe960a81dc43633881, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:56.489430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33ea5c-c6ad-473f-aab0-c8fd6a130b41 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:56.517029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 867/1762] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:56 2022] PUT /identity/v3/projects/f28391e42e2a4fe8b216495e3d6c68b7/users/e471de98af6545a3ba2f804be569cfc7/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:56.518546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb629a-ff29-45d3-80ce-68a2137e5c68 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.518822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb629a-ff29-45d3-80ce-68a2137e5c68 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.519095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb629a-ff29-45d3-80ce-68a2137e5c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.556445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 896/1763] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:56 2022] PUT /identity/v3/projects/396ec2cf84a34efe9cc84b30614db294/users/7cf8d956372b46fe960a81dc43633881/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:56.559819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33ea5c-c6ad-473f-aab0-c8fd6a130b41 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:56.570004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0b33ea5c-c6ad-473f-aab0-c8fd6a130b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:56.572254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ccdb629a-ff29-45d3-80ce-68a2137e5c68 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:56.573528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ccdb629a-ff29-45d3-80ce-68a2137e5c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:56.832077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33ea5c-c6ad-473f-aab0-c8fd6a130b41 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:56.841021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 868/1764] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:56.841021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb629a-ff29-45d3-80ce-68a2137e5c68 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:56.860177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb629a-ff29-45d3-80ce-68a2137e5c68 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:56.868366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100e553-e672-4e36-9720-1fa49a04613c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:56.869109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100e553-e672-4e36-9720-1fa49a04613c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.869439 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100e553-e672-4e36-9720-1fa49a04613c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.869716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100e553-e672-4e36-9720-1fa49a04613c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.870365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100e553-e672-4e36-9720-1fa49a04613c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:56.872216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100e553-e672-4e36-9720-1fa49a04613c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:56.879896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30180d-3755-4f81-a953-0cab5707a5b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:56.880554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30180d-3755-4f81-a953-0cab5707a5b9 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.880838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30180d-3755-4f81-a953-0cab5707a5b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.881114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30180d-3755-4f81-a953-0cab5707a5b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.895966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 869/1765] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:56.896910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100e553-e672-4e36-9720-1fa49a04613c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:56.913203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100e553-e672-4e36-9720-1fa49a04613c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:56.919919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69567c-ec46-4710-aefd-7663507ff705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:56.920566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69567c-ec46-4710-aefd-7663507ff705 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.920846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69567c-ec46-4710-aefd-7663507ff705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.921119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69567c-ec46-4710-aefd-7663507ff705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.921932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69567c-ec46-4710-aefd-7663507ff705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:56.923539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69567c-ec46-4710-aefd-7663507ff705 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:56.952842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 870/1766] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:56.954162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69567c-ec46-4710-aefd-7663507ff705 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:56.978424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69567c-ec46-4710-aefd-7663507ff705 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:56.991244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b575b40-5570-451c-94c3-f0452882b2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:56.992283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b575b40-5570-451c-94c3-f0452882b2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:56.992562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b575b40-5570-451c-94c3-f0452882b2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:56.992840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b575b40-5570-451c-94c3-f0452882b2b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:56.993496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b575b40-5570-451c-94c3-f0452882b2b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:20:56.993906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b575b40-5570-451c-94c3-f0452882b2b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:56.995401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b575b40-5570-451c-94c3-f0452882b2b6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:57.007363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:57.030732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 871/1767] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:20:57.031774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b575b40-5570-451c-94c3-f0452882b2b6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:57.063618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30180d-3755-4f81-a953-0cab5707a5b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:57.065219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30180d-3755-4f81-a953-0cab5707a5b9 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:57.072268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b575b40-5570-451c-94c3-f0452882b2b6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:57.092096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 897/1768] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:57.097009 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac042e8b-f8af-4413-8e35-8082f260a45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:57.097842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac042e8b-f8af-4413-8e35-8082f260a45b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:57.098337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac042e8b-f8af-4413-8e35-8082f260a45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:57.098588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac042e8b-f8af-4413-8e35-8082f260a45b admin admin] PATH_INFO: `/v3/projects/f28391e42e2a4fe8b216495e3d6c68b7/users/e471de98af6545a3ba2f804be569cfc7/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:57.104041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac042e8b-f8af-4413-8e35-8082f260a45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28391e42e2a4fe8b216495e3d6c68b7, user_id=e471de98af6545a3ba2f804be569cfc7, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:57.106221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30180d-3755-4f81-a953-0cab5707a5b9 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:57.118793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac042e8b-f8af-4413-8e35-8082f260a45b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:57.156872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 872/1769] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:57 2022] PUT /identity/v3/projects/f28391e42e2a4fe8b216495e3d6c68b7/users/e471de98af6545a3ba2f804be569cfc7/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:57.157886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac042e8b-f8af-4413-8e35-8082f260a45b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:57.158039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:57.170870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30180d-3755-4f81-a953-0cab5707a5b9 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:57.382975 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac042e8b-f8af-4413-8e35-8082f260a45b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:57.407516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7598cd9-2a2f-4963-8113-bd1058ef8107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:57.408198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7598cd9-2a2f-4963-8113-bd1058ef8107 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:57.408481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7598cd9-2a2f-4963-8113-bd1058ef8107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:57.408583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c3387-dee5-46ef-9fd4-6cff3d78ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:57.408766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7598cd9-2a2f-4963-8113-bd1058ef8107 admin admin] PATH_INFO: `/v3/projects/396ec2cf84a34efe9cc84b30614db294/users/7cf8d956372b46fe960a81dc43633881/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:57.409210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c3387-dee5-46ef-9fd4-6cff3d78ac38 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:57.409427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c3387-dee5-46ef-9fd4-6cff3d78ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:57.409696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c3387-dee5-46ef-9fd4-6cff3d78ac38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:57.410330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c3387-dee5-46ef-9fd4-6cff3d78ac38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:57.412255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c3387-dee5-46ef-9fd4-6cff3d78ac38 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:57.414908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7598cd9-2a2f-4963-8113-bd1058ef8107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=396ec2cf84a34efe9cc84b30614db294, user_id=7cf8d956372b46fe960a81dc43633881, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:57.417624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7598cd9-2a2f-4963-8113-bd1058ef8107 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:57.477041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 898/1770] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:57 2022] PUT /identity/v3/projects/396ec2cf84a34efe9cc84b30614db294/users/7cf8d956372b46fe960a81dc43633881/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:57.487873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe8e4a-46e5-424d-bcce-4e1596da9e55 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:57.487873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe8e4a-46e5-424d-bcce-4e1596da9e55 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:57.487873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe8e4a-46e5-424d-bcce-4e1596da9e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:57.502591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 873/1771] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:57 2022] POST /identity/v3/projects => generated 375 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:57.502591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcd186-52fc-46b6-a57b-9f5d161cb5f0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:57.504590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcd186-52fc-46b6-a57b-9f5d161cb5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:57.504590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcd186-52fc-46b6-a57b-9f5d161cb5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:57.550763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-79dcd186-52fc-46b6-a57b-9f5d161cb5f0 None None] MFA Rules not processed for user `e471de98af6545a3ba2f804be569cfc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:57.552851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-79dcd186-52fc-46b6-a57b-9f5d161cb5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:57.594142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5abe8e4a-46e5-424d-bcce-4e1596da9e55 None None] MFA Rules not processed for user `8790c086dcaa4da5b739bcd02d30b4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:57.598522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5abe8e4a-46e5-424d-bcce-4e1596da9e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:57.713641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 874/1772] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:57.714937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb874523-cc5f-4f11-bd27-0179ae20dd5c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:57.715219 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb874523-cc5f-4f11-bd27-0179ae20dd5c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:57.715490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb874523-cc5f-4f11-bd27-0179ae20dd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:57.776835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 899/1773] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:57.778483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe8e4a-46e5-424d-bcce-4e1596da9e55 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:57.789187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-eb874523-cc5f-4f11-bd27-0179ae20dd5c None None] MFA Rules not processed for user `7cf8d956372b46fe960a81dc43633881`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:57.789346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eb874523-cc5f-4f11-bd27-0179ae20dd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:57.916401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:57.925477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe8e4a-46e5-424d-bcce-4e1596da9e55 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:57.937576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0cc1f-e1c0-42fa-bc06-ca51866a2317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:57.938862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0cc1f-e1c0-42fa-bc06-ca51866a2317 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:57.939972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0cc1f-e1c0-42fa-bc06-ca51866a2317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:57.940538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0cc1f-e1c0-42fa-bc06-ca51866a2317 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:57.942366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0cc1f-e1c0-42fa-bc06-ca51866a2317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:57.946936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0cc1f-e1c0-42fa-bc06-ca51866a2317 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:57.994142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 875/1774] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:57.995790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f79fa-74cd-43d1-a9fb-f0c1f587a487 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:57.996289 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f79fa-74cd-43d1-a9fb-f0c1f587a487 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:57.996763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f79fa-74cd-43d1-a9fb-f0c1f587a487 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:57.999035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 876/1775] 10.222.0.71 () {54 vars in 912 bytes} [Wed Sep 21 12:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:20:58.001978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f79fa-74cd-43d1-a9fb-f0c1f587a487 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.007965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3e2f79fa-74cd-43d1-a9fb-f0c1f587a487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.032321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 900/1776] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:57 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:58.035238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0cc1f-e1c0-42fa-bc06-ca51866a2317 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.041198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-68d0cc1f-e1c0-42fa-bc06-ca51866a2317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.083345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0cc1f-e1c0-42fa-bc06-ca51866a2317 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.104696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.105452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.107276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.109449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] PATH_INFO: `/v3/users/5bc3315ec61549f5adeeadfe84392a66` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.113846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc3315ec61549f5adeeadfe84392a66)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:58.116838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:58.144960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:58.162456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f79fa-74cd-43d1-a9fb-f0c1f587a487 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.171630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c3a86-62ab-4da8-a097-bf093bbc2851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.172614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c3a86-62ab-4da8-a097-bf093bbc2851 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.173201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c3a86-62ab-4da8-a097-bf093bbc2851 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.173623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c3a86-62ab-4da8-a097-bf093bbc2851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.192010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c3a86-62ab-4da8-a097-bf093bbc2851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:58.193850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c3a86-62ab-4da8-a097-bf093bbc2851 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:58.207792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 877/1777] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:58.209873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671e925-5e2f-4c56-8bda-b841d7c2245f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.210238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671e925-5e2f-4c56-8bda-b841d7c2245f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.210728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671e925-5e2f-4c56-8bda-b841d7c2245f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.220736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc3315ec61549f5adeeadfe84392a66'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:58.234486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc3315ec61549f5adeeadfe84392a66'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:58.235639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc3315ec61549f5adeeadfe84392a66'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:58.238143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc3315ec61549f5adeeadfe84392a66'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:58.260464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc3315ec61549f5adeeadfe84392a66'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:20:58.273518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a671e925-5e2f-4c56-8bda-b841d7c2245f None None] MFA Rules not processed for user `cd66d5d9989449729cec7e60cdaf0949`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:58.273518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a671e925-5e2f-4c56-8bda-b841d7c2245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.298098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 901/1778] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:20:58 2022] DELETE /identity/v3/users/5bc3315ec61549f5adeeadfe84392a66 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:58.298098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.302708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.367273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 878/1779] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:58.368341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671e925-5e2f-4c56-8bda-b841d7c2245f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.371542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a671e925-5e2f-4c56-8bda-b841d7c2245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.392911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671e925-5e2f-4c56-8bda-b841d7c2245f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.414917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801c74-b20e-43c7-bebf-9c45b2be426e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.421265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2e844-d987-4249-ad7b-467f5cc1cc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.422053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e844-d987-4249-ad7b-467f5cc1cc30 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.422786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e844-d987-4249-ad7b-467f5cc1cc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.423383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2e844-d987-4249-ad7b-467f5cc1cc30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.424267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2e844-d987-4249-ad7b-467f5cc1cc30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:58.426186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2e844-d987-4249-ad7b-467f5cc1cc30 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:58.444098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 902/1780] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:58.445978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c6477-3030-4d0f-b25e-8a3897f095d0 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.446285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c6477-3030-4d0f-b25e-8a3897f095d0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.446887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c6477-3030-4d0f-b25e-8a3897f095d0 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.455066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 903/1781] 10.222.0.71 () {54 vars in 912 bytes} [Wed Sep 21 12:20:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:20:58.457149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3948b07-cf36-44d4-9f2d-9dc73719463e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.457341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3948b07-cf36-44d4-9f2d-9dc73719463e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.457615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3948b07-cf36-44d4-9f2d-9dc73719463e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.483158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.483776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.484377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.484772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.486858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.497596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f3948b07-cf36-44d4-9f2d-9dc73719463e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:58.499558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f3948b07-cf36-44d4-9f2d-9dc73719463e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.513423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 904/1782] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:58.513423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3948b07-cf36-44d4-9f2d-9dc73719463e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.522874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f3948b07-cf36-44d4-9f2d-9dc73719463e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.544599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3948b07-cf36-44d4-9f2d-9dc73719463e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.550703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:58.561456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce860bd-c27b-4436-9b3a-af0df80511b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.564470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce860bd-c27b-4436-9b3a-af0df80511b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.567514 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce860bd-c27b-4436-9b3a-af0df80511b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.567514 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce860bd-c27b-4436-9b3a-af0df80511b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.571624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9ce860bd-c27b-4436-9b3a-af0df80511b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.606931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:58.609239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:58.621134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 879/1783] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:58.622211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.624578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.678676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce860bd-c27b-4436-9b3a-af0df80511b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:58.680775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce860bd-c27b-4436-9b3a-af0df80511b0 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:58.697791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 905/1784] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:58.699156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce860bd-c27b-4436-9b3a-af0df80511b0 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.713543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce860bd-c27b-4436-9b3a-af0df80511b0 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.720351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9739f-1a6d-4912-b6f5-03667fd1d2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.721191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9739f-1a6d-4912-b6f5-03667fd1d2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.722168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9739f-1a6d-4912-b6f5-03667fd1d2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.722168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9739f-1a6d-4912-b6f5-03667fd1d2c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.722847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9739f-1a6d-4912-b6f5-03667fd1d2c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:58.727122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9739f-1a6d-4912-b6f5-03667fd1d2c8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:58.744277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19851e-7aa3-410b-b29d-1c35f1cfa5d8 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.749319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1aa0f-1fa3-42e3-a7a0-62c9014463bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.751690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1aa0f-1fa3-42e3-a7a0-62c9014463bb service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.752210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1aa0f-1fa3-42e3-a7a0-62c9014463bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.752679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1aa0f-1fa3-42e3-a7a0-62c9014463bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.770626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 906/1785] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:20:58.772327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b1f6d-79cc-48fd-b002-cf42eef07740 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.772792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b1f6d-79cc-48fd-b002-cf42eef07740 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.773199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b1f6d-79cc-48fd-b002-cf42eef07740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.780813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1aa0f-1fa3-42e3-a7a0-62c9014463bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:58.783316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1aa0f-1fa3-42e3-a7a0-62c9014463bb service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:58.799307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 880/1786] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:58.799831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:20:58.801097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1aa0f-1fa3-42e3-a7a0-62c9014463bb service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.804816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-64c1aa0f-1fa3-42e3-a7a0-62c9014463bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.817761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c06b1f6d-79cc-48fd-b002-cf42eef07740 None None] MFA Rules not processed for user `cd66d5d9989449729cec7e60cdaf0949`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:58.819793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c06b1f6d-79cc-48fd-b002-cf42eef07740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:58.850387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1aa0f-1fa3-42e3-a7a0-62c9014463bb service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.853792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 907/1787] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:58.855393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b1f6d-79cc-48fd-b002-cf42eef07740 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.876736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8517ee-9cbc-42e1-86a9-31a8d0377781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.877390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8517ee-9cbc-42e1-86a9-31a8d0377781 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.877677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8517ee-9cbc-42e1-86a9-31a8d0377781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.877956 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8517ee-9cbc-42e1-86a9-31a8d0377781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.889779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b1f6d-79cc-48fd-b002-cf42eef07740 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.902649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847abec-f644-4474-a966-b401d04c0cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.903563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847abec-f644-4474-a966-b401d04c0cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.905418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8517ee-9cbc-42e1-86a9-31a8d0377781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:58.906478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847abec-f644-4474-a966-b401d04c0cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.906478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847abec-f644-4474-a966-b401d04c0cd6 admin admin] PATH_INFO: `/v3/projects/fee0e87f87b44bdba46c0b23f3e58ce6/users/5cbc407e89d044698fb4c12112234542/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:58.916616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8517ee-9cbc-42e1-86a9-31a8d0377781 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:58.940678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847abec-f644-4474-a966-b401d04c0cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee0e87f87b44bdba46c0b23f3e58ce6, user_id=5cbc407e89d044698fb4c12112234542, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:58.945324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847abec-f644-4474-a966-b401d04c0cd6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:58.948245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 881/1788] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:58.951369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8517ee-9cbc-42e1-86a9-31a8d0377781 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:58.972865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8517ee-9cbc-42e1-86a9-31a8d0377781 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:58.990104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dabfb6-481f-4cb3-853b-b642d4d165a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:58.993597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dabfb6-481f-4cb3-853b-b642d4d165a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:58.994121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dabfb6-481f-4cb3-853b-b642d4d165a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:58.995724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dabfb6-481f-4cb3-853b-b642d4d165a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:59.002478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 908/1789] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:20:58 2022] PUT /identity/v3/projects/fee0e87f87b44bdba46c0b23f3e58ce6/users/5cbc407e89d044698fb4c12112234542/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:20:59.015303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d9582-5d44-418d-81ff-64d9a6c1a541 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:59.015799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d9582-5d44-418d-81ff-64d9a6c1a541 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:59.016171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d9582-5d44-418d-81ff-64d9a6c1a541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:59.024747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dabfb6-481f-4cb3-853b-b642d4d165a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:59.031798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dabfb6-481f-4cb3-853b-b642d4d165a3 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:59.057699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 882/1790] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:20:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:20:59.113501 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-498d9582-5d44-418d-81ff-64d9a6c1a541 None None] MFA Rules not processed for user `5cbc407e89d044698fb4c12112234542`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:20:59.118939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-498d9582-5d44-418d-81ff-64d9a6c1a541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:59.264086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 909/1791] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:20:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:20:59.268217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dabfb6-481f-4cb3-853b-b642d4d165a3 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:59.296632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dabfb6-481f-4cb3-853b-b642d4d165a3 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:59.541045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820ee2a-99dc-48d3-ae78-f7e185ea3879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:59.544333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820ee2a-99dc-48d3-ae78-f7e185ea3879 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:59.545880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820ee2a-99dc-48d3-ae78-f7e185ea3879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:59.546170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820ee2a-99dc-48d3-ae78-f7e185ea3879 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:59.550819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820ee2a-99dc-48d3-ae78-f7e185ea3879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:20:59.556423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820ee2a-99dc-48d3-ae78-f7e185ea3879 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:20:59.632942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 883/1792] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:20:59 2022] POST /identity/v3/projects => generated 377 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:20:59.637149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d9582-5d44-418d-81ff-64d9a6c1a541 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:59.659179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d9582-5d44-418d-81ff-64d9a6c1a541 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:20:59.932755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820ee2a-99dc-48d3-ae78-f7e185ea3879 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:20:59.940469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9820ee2a-99dc-48d3-ae78-f7e185ea3879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:20:59.996263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0deec8-765d-4330-bdba-a61aa786dbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:20:59.996924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0deec8-765d-4330-bdba-a61aa786dbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:20:59.997208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0deec8-765d-4330-bdba-a61aa786dbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:20:59.997522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0deec8-765d-4330-bdba-a61aa786dbf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:20:59.998683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0deec8-765d-4330-bdba-a61aa786dbf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:00.015600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0deec8-765d-4330-bdba-a61aa786dbf7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:00.029637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820ee2a-99dc-48d3-ae78-f7e185ea3879 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:00.077584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:00.086870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:00.092620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:00.097419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] PATH_INFO: `/v3/projects/5f22fbc5daa94080bebad8757df9655e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:00.106299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f22fbc5daa94080bebad8757df9655e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:00.109025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:00.216289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 910/1793] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:20:59 2022] POST /identity/v3/users => generated 295 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:00.217874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0deec8-765d-4330-bdba-a61aa786dbf7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:00.235671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6c0deec8-765d-4330-bdba-a61aa786dbf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:00.295990 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:00.352492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f22fbc5daa94080bebad8757df9655e', 'request_id': 'req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:00.375063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f22fbc5daa94080bebad8757df9655e', 'request_id': 'req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:00.385388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f22fbc5daa94080bebad8757df9655e', 'request_id': 'req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:00.403008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] The token cache is being invalidate because project 5f22fbc5daa94080bebad8757df9655e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:00.404902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:00.406049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:00.409866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 884/1794] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:20:59 2022] DELETE /identity/v3/projects/5f22fbc5daa94080bebad8757df9655e => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:00.411490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:00.419318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:00.436857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:00.496258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0deec8-765d-4330-bdba-a61aa786dbf7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:00.656707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955ba8a-6af3-4f9a-8dbf-baba6c0cb6f5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:00.679824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88e26b-23ae-49f1-ae5f-ef282ea7b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:00.682075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88e26b-23ae-49f1-ae5f-ef282ea7b335 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:00.682761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88e26b-23ae-49f1-ae5f-ef282ea7b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:00.685109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88e26b-23ae-49f1-ae5f-ef282ea7b335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:00.686488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88e26b-23ae-49f1-ae5f-ef282ea7b335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:00.689929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88e26b-23ae-49f1-ae5f-ef282ea7b335 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:00.706383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f84a-7244-47df-ab4c-bf4cfb2c7eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:00.707254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f84a-7244-47df-ab4c-bf4cfb2c7eb9 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:00.707678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f84a-7244-47df-ab4c-bf4cfb2c7eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:00.708378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f84a-7244-47df-ab4c-bf4cfb2c7eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:00.710362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8c58f84a-7244-47df-ab4c-bf4cfb2c7eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:00.718688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 885/1795] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:00.719981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88e26b-23ae-49f1-ae5f-ef282ea7b335 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:00.725093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0d88e26b-23ae-49f1-ae5f-ef282ea7b335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:00.835624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58f84a-7244-47df-ab4c-bf4cfb2c7eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:00.837676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58f84a-7244-47df-ab4c-bf4cfb2c7eb9 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:00.852994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 911/1796] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:00.857675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58a1af-1b33-46d0-8167-731a010f4440 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:00.858100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58a1af-1b33-46d0-8167-731a010f4440 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:00.858381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58a1af-1b33-46d0-8167-731a010f4440 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:00.860192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 912/1797] 10.222.0.71 () {54 vars in 961 bytes} [Wed Sep 21 12:21:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:21:00.861266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58a1af-1b33-46d0-8167-731a010f4440 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:00.863640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8a58a1af-1b33-46d0-8167-731a010f4440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:00.894047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58a1af-1b33-46d0-8167-731a010f4440 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:00.910891 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:00.911856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:00.912087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:00.912303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] PATH_INFO: `/v3/users/35fb1abceb0c4459a2336f4c973e5a5d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:00.913503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35fb1abceb0c4459a2336f4c973e5a5d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:00.916249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:00.968992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88e26b-23ae-49f1-ae5f-ef282ea7b335 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:00.976114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3b358-76ce-42a9-a194-7867274aca05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:00.976789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3b358-76ce-42a9-a194-7867274aca05 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:00.977105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3b358-76ce-42a9-a194-7867274aca05 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:00.977728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3b358-76ce-42a9-a194-7867274aca05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:00.986835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0ba3b358-76ce-42a9-a194-7867274aca05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:01.026112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3b358-76ce-42a9-a194-7867274aca05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:01.028230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3b358-76ce-42a9-a194-7867274aca05 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:01.068149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 886/1798] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:01.068979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3b358-76ce-42a9-a194-7867274aca05 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:01.076947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35fb1abceb0c4459a2336f4c973e5a5d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:01.106089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35fb1abceb0c4459a2336f4c973e5a5d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:01.109669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35fb1abceb0c4459a2336f4c973e5a5d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:01.110295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35fb1abceb0c4459a2336f4c973e5a5d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:01.121868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3b358-76ce-42a9-a194-7867274aca05 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:01.145516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c7263b4c-960b-4383-b9bc-6a0e838d4d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35fb1abceb0c4459a2336f4c973e5a5d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:01.145708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd808702-8dfd-4e64-a863-c2fa093cf4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:01.146461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd808702-8dfd-4e64-a863-c2fa093cf4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:01.146753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd808702-8dfd-4e64-a863-c2fa093cf4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:01.147063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd808702-8dfd-4e64-a863-c2fa093cf4f0 admin admin] PATH_INFO: `/v3/projects/9d075708835c4e2c9df8905d66b779af/users/24fb492aa29845e6add5951b4214a0b8/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:01.202077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd808702-8dfd-4e64-a863-c2fa093cf4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d075708835c4e2c9df8905d66b779af, user_id=24fb492aa29845e6add5951b4214a0b8, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:01.209785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 913/1799] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:00 2022] DELETE /identity/v3/users/35fb1abceb0c4459a2336f4c973e5a5d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:01.223964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5cd69-60b5-4bdc-9e86-3cde8239cf96 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:01.229652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd808702-8dfd-4e64-a863-c2fa093cf4f0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:01.233293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5cd69-60b5-4bdc-9e86-3cde8239cf96 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:01.233647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b5cd69-60b5-4bdc-9e86-3cde8239cf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:01.295482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 887/1800] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:01 2022] PUT /identity/v3/projects/9d075708835c4e2c9df8905d66b779af/users/24fb492aa29845e6add5951b4214a0b8/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:01.301132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd808702-8dfd-4e64-a863-c2fa093cf4f0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:01.305517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:01.306765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fd808702-8dfd-4e64-a863-c2fa093cf4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:01.342656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a9b5cd69-60b5-4bdc-9e86-3cde8239cf96 None None] MFA Rules not processed for user `eac29fd43f1f4bbd987950b90385200b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:01.361068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a9b5cd69-60b5-4bdc-9e86-3cde8239cf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:01.399200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:01.519152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 914/1801] 10.222.0.71 () {58 vars in 1058 bytes} [Wed Sep 21 12:21:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:01.528783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5cd69-60b5-4bdc-9e86-3cde8239cf96 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:01.581458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b5cd69-60b5-4bdc-9e86-3cde8239cf96 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:01.596529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd808702-8dfd-4e64-a863-c2fa093cf4f0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:01.599662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae0a12-303c-4d71-9b65-1cacdac1f87e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:01.600792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae0a12-303c-4d71-9b65-1cacdac1f87e service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:01.601390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae0a12-303c-4d71-9b65-1cacdac1f87e service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:01.602351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae0a12-303c-4d71-9b65-1cacdac1f87e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:01.608828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fcae0a12-303c-4d71-9b65-1cacdac1f87e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:01.611322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502f166-88e8-43c6-ad60-81cf5f5074d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:01.612589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502f166-88e8-43c6-ad60-81cf5f5074d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:01.613845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502f166-88e8-43c6-ad60-81cf5f5074d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:01.614435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502f166-88e8-43c6-ad60-81cf5f5074d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:01.617325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502f166-88e8-43c6-ad60-81cf5f5074d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:01.620741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502f166-88e8-43c6-ad60-81cf5f5074d6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:01.641151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae0a12-303c-4d71-9b65-1cacdac1f87e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:01.647012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae0a12-303c-4d71-9b65-1cacdac1f87e service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:01.648848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 888/1802] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:01.655800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502f166-88e8-43c6-ad60-81cf5f5074d6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:01.693518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502f166-88e8-43c6-ad60-81cf5f5074d6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:01.694304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 915/1803] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:01.708825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcae60-bf13-4a2a-b917-1ee8ef9e6c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:01.712117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcae60-bf13-4a2a-b917-1ee8ef9e6c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:01.714397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcae60-bf13-4a2a-b917-1ee8ef9e6c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:01.715713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcae60-bf13-4a2a-b917-1ee8ef9e6c84 admin admin] PATH_INFO: `/v3/projects/9d075708835c4e2c9df8905d66b779af/users/24fb492aa29845e6add5951b4214a0b8/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:01.721533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcae60-bf13-4a2a-b917-1ee8ef9e6c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d075708835c4e2c9df8905d66b779af, user_id=24fb492aa29845e6add5951b4214a0b8, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:01.730160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcae60-bf13-4a2a-b917-1ee8ef9e6c84 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:01.781078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 889/1804] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:01 2022] PUT /identity/v3/projects/9d075708835c4e2c9df8905d66b779af/users/24fb492aa29845e6add5951b4214a0b8/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:01.786565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ad84b-57cc-4d16-ae38-c5d2dd37aca1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:01.787176 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ad84b-57cc-4d16-ae38-c5d2dd37aca1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:01.788100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ad84b-57cc-4d16-ae38-c5d2dd37aca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:01.869221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2f2ad84b-57cc-4d16-ae38-c5d2dd37aca1 None None] MFA Rules not processed for user `24fb492aa29845e6add5951b4214a0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:01.876594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f2ad84b-57cc-4d16-ae38-c5d2dd37aca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:02.056887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcae60-bf13-4a2a-b917-1ee8ef9e6c84 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:02.059236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-41bcae60-bf13-4a2a-b917-1ee8ef9e6c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:02.132080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 916/1805] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:02.150818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ad84b-57cc-4d16-ae38-c5d2dd37aca1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:02.157098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2f2ad84b-57cc-4d16-ae38-c5d2dd37aca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:02.285755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcae60-bf13-4a2a-b917-1ee8ef9e6c84 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:02.291254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:02.292106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:02.292385 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:02.292658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] PATH_INFO: `/v3/users/f7a2802c5a42420c8d959be3bcc88c07` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:02.295702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7a2802c5a42420c8d959be3bcc88c07)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:02.299227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:02.409642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7a2802c5a42420c8d959be3bcc88c07'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:02.414951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7a2802c5a42420c8d959be3bcc88c07'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:02.439846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7a2802c5a42420c8d959be3bcc88c07'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:02.440416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7a2802c5a42420c8d959be3bcc88c07'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:02.455211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ad84b-57cc-4d16-ae38-c5d2dd37aca1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:02.467665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00470999-d376-49e2-8779-19338d3de1ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:02.468901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7a2802c5a42420c8d959be3bcc88c07'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:02.471104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00470999-d376-49e2-8779-19338d3de1ee service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:02.471409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00470999-d376-49e2-8779-19338d3de1ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:02.472042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00470999-d376-49e2-8779-19338d3de1ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:02.481588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-00470999-d376-49e2-8779-19338d3de1ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:02.551700 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 890/1806] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:02 2022] DELETE /identity/v3/users/f7a2802c5a42420c8d959be3bcc88c07 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:02.555873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:02.557614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:02.559202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:02.652333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:02.667729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00470999-d376-49e2-8779-19338d3de1ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:02.671702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00470999-d376-49e2-8779-19338d3de1ee service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:02.721780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 917/1807] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:02.737722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa0c71-887c-4511-a00b-76e619b90561 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:02.741029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa0c71-887c-4511-a00b-76e619b90561 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:02.741320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa0c71-887c-4511-a00b-76e619b90561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:02.812052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-effa0c71-887c-4511-a00b-76e619b90561 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:02.816856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-effa0c71-887c-4511-a00b-76e619b90561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:02.819468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc7a99-3645-475e-b167-5d827b7983f0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:02.833999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:02.834807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:02.835199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:02.835589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] PATH_INFO: `/v3/projects/17b036ef79454dc1afd752c2e155bd18` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:02.837908 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=17b036ef79454dc1afd752c2e155bd18)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:02.840230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:02.866740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 918/1808] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:02.882808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa0c71-887c-4511-a00b-76e619b90561 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:02.887361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-effa0c71-887c-4511-a00b-76e619b90561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:03.028438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17b036ef79454dc1afd752c2e155bd18', 'request_id': 'req-65e7fc26-f189-4761-b44f-296c287e95fd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:03.049660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17b036ef79454dc1afd752c2e155bd18', 'request_id': 'req-65e7fc26-f189-4761-b44f-296c287e95fd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:03.073827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17b036ef79454dc1afd752c2e155bd18', 'request_id': 'req-65e7fc26-f189-4761-b44f-296c287e95fd'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:03.104533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] The token cache is being invalidate because project 17b036ef79454dc1afd752c2e155bd18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:03.106718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:03.110747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-65e7fc26-f189-4761-b44f-296c287e95fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:03.116829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 891/1809] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:02 2022] DELETE /identity/v3/projects/17b036ef79454dc1afd752c2e155bd18 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:03.123407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa0c71-887c-4511-a00b-76e619b90561 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:03.129895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b33a48-90f5-409d-a0b4-606934fe98a5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:03.132092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b33a48-90f5-409d-a0b4-606934fe98a5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:03.140035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b33a48-90f5-409d-a0b4-606934fe98a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:03.200061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-94b33a48-90f5-409d-a0b4-606934fe98a5 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:03.205669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-94b33a48-90f5-409d-a0b4-606934fe98a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:03.361469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268f3f1-55b9-4b49-8af5-88ff5799d0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:03.366921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268f3f1-55b9-4b49-8af5-88ff5799d0dd service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:03.367442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268f3f1-55b9-4b49-8af5-88ff5799d0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:03.367946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268f3f1-55b9-4b49-8af5-88ff5799d0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:03.372440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f268f3f1-55b9-4b49-8af5-88ff5799d0dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:03.413749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 892/1810] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:03.420595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be646f4-83bb-42b5-8c3d-5233cca8651b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:03.421012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be646f4-83bb-42b5-8c3d-5233cca8651b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:03.421301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be646f4-83bb-42b5-8c3d-5233cca8651b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:03.468893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3be646f4-83bb-42b5-8c3d-5233cca8651b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:03.471363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3be646f4-83bb-42b5-8c3d-5233cca8651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:03.492621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 893/1811] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:03.494825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be646f4-83bb-42b5-8c3d-5233cca8651b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:03.531443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be646f4-83bb-42b5-8c3d-5233cca8651b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:03.548278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8ce8f-ca90-4311-8486-145b600d1346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:03.549050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8ce8f-ca90-4311-8486-145b600d1346 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:03.553427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8ce8f-ca90-4311-8486-145b600d1346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:03.554406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8ce8f-ca90-4311-8486-145b600d1346 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:03.556015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8ce8f-ca90-4311-8486-145b600d1346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:21:03.559669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8ce8f-ca90-4311-8486-145b600d1346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:03.561592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268f3f1-55b9-4b49-8af5-88ff5799d0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:03.564856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268f3f1-55b9-4b49-8af5-88ff5799d0dd service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:03.565153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8ce8f-ca90-4311-8486-145b600d1346 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:03.598315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 919/1812] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:21:03.604619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268f3f1-55b9-4b49-8af5-88ff5799d0dd service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:03.605484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 894/1813] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:21:03.617337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268f3f1-55b9-4b49-8af5-88ff5799d0dd service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:03.642546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2868018-8610-4c17-aa4b-011f130c66f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:03.644747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2868018-8610-4c17-aa4b-011f130c66f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:03.667321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2868018-8610-4c17-aa4b-011f130c66f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:03.667887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2868018-8610-4c17-aa4b-011f130c66f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:03.669284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2868018-8610-4c17-aa4b-011f130c66f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:03.672572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2868018-8610-4c17-aa4b-011f130c66f6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:03.831270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8ce8f-ca90-4311-8486-145b600d1346 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:03.845561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 920/1814] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:03 2022] POST /identity/v3/projects => generated 399 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:03.848800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8ce8f-ca90-4311-8486-145b600d1346 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:03.927663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:04.010918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd9b75-ca78-4330-a0d8-dd7be04ab6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.012012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd9b75-ca78-4330-a0d8-dd7be04ab6f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.012948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd9b75-ca78-4330-a0d8-dd7be04ab6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.013227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd9b75-ca78-4330-a0d8-dd7be04ab6f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.014559 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd9b75-ca78-4330-a0d8-dd7be04ab6f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.020848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd9b75-ca78-4330-a0d8-dd7be04ab6f9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.069188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2868018-8610-4c17-aa4b-011f130c66f6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.077124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:04.077534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 895/1815] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:03 2022] POST /identity/v3/users => generated 300 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:04.086163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2868018-8610-4c17-aa4b-011f130c66f6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:04.100856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e32c7-1873-41dc-a7a2-f894d810621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.102188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e32c7-1873-41dc-a7a2-f894d810621d admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.102777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e32c7-1873-41dc-a7a2-f894d810621d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.103465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e32c7-1873-41dc-a7a2-f894d810621d admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.104591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e32c7-1873-41dc-a7a2-f894d810621d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.106618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e32c7-1873-41dc-a7a2-f894d810621d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.124628 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 921/1816] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:04.126316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd9b75-ca78-4330-a0d8-dd7be04ab6f9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.141615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd9b75-ca78-4330-a0d8-dd7be04ab6f9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:04.148133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f670001-c0be-4703-964b-c2b730cbc227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.148766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f670001-c0be-4703-964b-c2b730cbc227 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.149468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f670001-c0be-4703-964b-c2b730cbc227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.149760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f670001-c0be-4703-964b-c2b730cbc227 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.150626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f670001-c0be-4703-964b-c2b730cbc227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.152973 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f670001-c0be-4703-964b-c2b730cbc227 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.171330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 896/1817] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:04.171953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e32c7-1873-41dc-a7a2-f894d810621d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.184250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e32c7-1873-41dc-a7a2-f894d810621d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:04.190665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b97ed-427e-48b7-afe5-d7898ac8ef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.191355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b97ed-427e-48b7-afe5-d7898ac8ef41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.191635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b97ed-427e-48b7-afe5-d7898ac8ef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.203277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b97ed-427e-48b7-afe5-d7898ac8ef41 admin admin] PATH_INFO: `/v3/projects/a337d07daf7740b1b9fb7062a7510153/users/bd7b747c782e41f893f9c45db1494e1b/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.238965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b97ed-427e-48b7-afe5-d7898ac8ef41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a337d07daf7740b1b9fb7062a7510153, user_id=bd7b747c782e41f893f9c45db1494e1b, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.241875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b97ed-427e-48b7-afe5-d7898ac8ef41 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.283906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 922/1818] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:04 2022] PUT /identity/v3/projects/a337d07daf7740b1b9fb7062a7510153/users/bd7b747c782e41f893f9c45db1494e1b/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:04.291947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f22995-0027-4e25-9842-a1cd3a71bab8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.292237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f22995-0027-4e25-9842-a1cd3a71bab8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.292510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f22995-0027-4e25-9842-a1cd3a71bab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.306113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b97ed-427e-48b7-afe5-d7898ac8ef41 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.308813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0e2b97ed-427e-48b7-afe5-d7898ac8ef41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:04.325498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-08f22995-0027-4e25-9842-a1cd3a71bab8 None None] MFA Rules not processed for user `bd7b747c782e41f893f9c45db1494e1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:04.327324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-08f22995-0027-4e25-9842-a1cd3a71bab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:04.411146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b97ed-427e-48b7-afe5-d7898ac8ef41 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:04.416934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.417901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.418345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.419005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] PATH_INFO: `/v3/projects/ba887b47f56c472997515ee30031246e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.422429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba887b47f56c472997515ee30031246e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.424870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.428218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 897/1819] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:04.433055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f22995-0027-4e25-9842-a1cd3a71bab8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.450475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f22995-0027-4e25-9842-a1cd3a71bab8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:04.457575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626fff0-b176-4469-b54b-745cdf604c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.458566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626fff0-b176-4469-b54b-745cdf604c50 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.458919 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626fff0-b176-4469-b54b-745cdf604c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.459265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626fff0-b176-4469-b54b-745cdf604c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.460016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626fff0-b176-4469-b54b-745cdf604c50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.462066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626fff0-b176-4469-b54b-745cdf604c50 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.500452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 898/1820] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:04 2022] POST /identity/v3/projects => generated 401 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:04.505067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626fff0-b176-4469-b54b-745cdf604c50 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.508903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba887b47f56c472997515ee30031246e', 'request_id': 'req-bfbd0de0-d056-47f3-bfd4-46b0b200450f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.519476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba887b47f56c472997515ee30031246e', 'request_id': 'req-bfbd0de0-d056-47f3-bfd4-46b0b200450f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.538347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba887b47f56c472997515ee30031246e', 'request_id': 'req-bfbd0de0-d056-47f3-bfd4-46b0b200450f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.567808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] The token cache is being invalidate because project ba887b47f56c472997515ee30031246e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:04.568920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.570378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.574112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 923/1821] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:04 2022] DELETE /identity/v3/projects/ba887b47f56c472997515ee30031246e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:21:04.584988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.586913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:04.590623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626fff0-b176-4469-b54b-745cdf604c50 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:04.595946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f68d76-acb9-4020-84cd-a69e7346f333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.596638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f68d76-acb9-4020-84cd-a69e7346f333 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.596986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f68d76-acb9-4020-84cd-a69e7346f333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.597330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f68d76-acb9-4020-84cd-a69e7346f333 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.598065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f68d76-acb9-4020-84cd-a69e7346f333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.600280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f68d76-acb9-4020-84cd-a69e7346f333 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.628709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 899/1822] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:04 2022] POST /identity/v3/users => generated 307 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:04.632711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f68d76-acb9-4020-84cd-a69e7346f333 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.635172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-02f68d76-acb9-4020-84cd-a69e7346f333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:04.648806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:04.658304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f68d76-acb9-4020-84cd-a69e7346f333 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:04.665937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2cc4a-3bd9-4f0d-811c-a65eb82ef6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.666775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2cc4a-3bd9-4f0d-811c-a65eb82ef6dd admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.667206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2cc4a-3bd9-4f0d-811c-a65eb82ef6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.667551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2cc4a-3bd9-4f0d-811c-a65eb82ef6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.668410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2cc4a-3bd9-4f0d-811c-a65eb82ef6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.670250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2cc4a-3bd9-4f0d-811c-a65eb82ef6dd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.682912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0de0-d056-47f3-bfd4-46b0b200450f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:04.687783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:04.687929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 900/1823] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:04.689939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2cc4a-3bd9-4f0d-811c-a65eb82ef6dd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.693405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.694915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.695494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.696021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] PATH_INFO: `/v3/users/7cf8d956372b46fe960a81dc43633881` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.697443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cf8d956372b46fe960a81dc43633881)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.699796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.703395 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2cc4a-3bd9-4f0d-811c-a65eb82ef6dd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:04.710817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f6bb-4d5c-422d-9581-9bfc4df8fbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:04.711492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f6bb-4d5c-422d-9581-9bfc4df8fbd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.711793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f6bb-4d5c-422d-9581-9bfc4df8fbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.712073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4f6bb-4d5c-422d-9581-9bfc4df8fbd9 admin admin] PATH_INFO: `/v3/projects/74d1c3287f3b497d8d5f8a3969db642e/users/7b8afab10f8e4f85b73a6a5a5b763136/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.745466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4f6bb-4d5c-422d-9581-9bfc4df8fbd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d1c3287f3b497d8d5f8a3969db642e, user_id=7b8afab10f8e4f85b73a6a5a5b763136, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:04.745466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4f6bb-4d5c-422d-9581-9bfc4df8fbd9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:04.768594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 901/1824] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:04 2022] PUT /identity/v3/projects/74d1c3287f3b497d8d5f8a3969db642e/users/7b8afab10f8e4f85b73a6a5a5b763136/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:21:04.781444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f6bb-4d5c-422d-9581-9bfc4df8fbd9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:04.824107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cf8d956372b46fe960a81dc43633881'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.832207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cf8d956372b46fe960a81dc43633881'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.833224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cf8d956372b46fe960a81dc43633881'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.833918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cf8d956372b46fe960a81dc43633881'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.850829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bfab6ebe-4286-4c43-be75-ad78439320a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cf8d956372b46fe960a81dc43633881'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:04.890667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 924/1825] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:04 2022] DELETE /identity/v3/users/7cf8d956372b46fe960a81dc43633881 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:04.892106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a3f21-dc28-4834-b665-d9e5f920b773 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:04.892392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a3f21-dc28-4834-b665-d9e5f920b773 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:04.892705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a3f21-dc28-4834-b665-d9e5f920b773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:04.912726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-240a3f21-dc28-4834-b665-d9e5f920b773 None None] MFA Rules not processed for user `8790c086dcaa4da5b739bcd02d30b4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:04.913960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-240a3f21-dc28-4834-b665-d9e5f920b773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:04.917031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4f6bb-4d5c-422d-9581-9bfc4df8fbd9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:05.003395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 925/1826] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:05.021152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ea794-206c-4a33-825a-e7e5f5cd3a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:05.023048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ea794-206c-4a33-825a-e7e5f5cd3a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.024234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ea794-206c-4a33-825a-e7e5f5cd3a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.024723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ea794-206c-4a33-825a-e7e5f5cd3a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.026415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ea794-206c-4a33-825a-e7e5f5cd3a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:05.029202 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ea794-206c-4a33-825a-e7e5f5cd3a72 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:05.058684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a3f21-dc28-4834-b665-d9e5f920b773 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:05.060172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-240a3f21-dc28-4834-b665-d9e5f920b773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:05.064486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 902/1827] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:05.076241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a3f21-dc28-4834-b665-d9e5f920b773 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:05.081482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5852d2-4e88-4535-b3d1-240760500c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:05.082348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5852d2-4e88-4535-b3d1-240760500c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.083288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5852d2-4e88-4535-b3d1-240760500c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.083288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5852d2-4e88-4535-b3d1-240760500c84 admin admin] PATH_INFO: `/v3/projects/74d1c3287f3b497d8d5f8a3969db642e/users/7b8afab10f8e4f85b73a6a5a5b763136/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.085531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5852d2-4e88-4535-b3d1-240760500c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d1c3287f3b497d8d5f8a3969db642e, user_id=7b8afab10f8e4f85b73a6a5a5b763136, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:05.088502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5852d2-4e88-4535-b3d1-240760500c84 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:05.116315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 926/1828] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:05 2022] PUT /identity/v3/projects/74d1c3287f3b497d8d5f8a3969db642e/users/7b8afab10f8e4f85b73a6a5a5b763136/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:05.121535 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca8460-f542-4810-a5c6-0abc9d8b01b0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.121688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca8460-f542-4810-a5c6-0abc9d8b01b0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.121932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca8460-f542-4810-a5c6-0abc9d8b01b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.164961 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a4ca8460-f542-4810-a5c6-0abc9d8b01b0 None None] MFA Rules not processed for user `7b8afab10f8e4f85b73a6a5a5b763136`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:05.166375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a4ca8460-f542-4810-a5c6-0abc9d8b01b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:05.253599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:05.271542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 903/1829] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:05.280080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5852d2-4e88-4535-b3d1-240760500c84 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:05.282264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb5852d2-4e88-4535-b3d1-240760500c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:05.283845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca8460-f542-4810-a5c6-0abc9d8b01b0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:05.286076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a4ca8460-f542-4810-a5c6-0abc9d8b01b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:05.360643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca8460-f542-4810-a5c6-0abc9d8b01b0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:05.363399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5852d2-4e88-4535-b3d1-240760500c84 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:05.364940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093de41e-3207-4c72-8aa5-e82699182ac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:05.365607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093de41e-3207-4c72-8aa5-e82699182ac3 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.365949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093de41e-3207-4c72-8aa5-e82699182ac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.366287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093de41e-3207-4c72-8aa5-e82699182ac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.368374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:05.369080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.369369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.369648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] PATH_INFO: `/v3/projects/396ec2cf84a34efe9cc84b30614db294` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.370816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=396ec2cf84a34efe9cc84b30614db294)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:05.373389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:05.377943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093de41e-3207-4c72-8aa5-e82699182ac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:05.380118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093de41e-3207-4c72-8aa5-e82699182ac3 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:05.385615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:05.413297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 904/1830] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:05 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:05.445174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '396ec2cf84a34efe9cc84b30614db294', 'request_id': 'req-d5113621-c6be-4198-b465-9c31f05a29e1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:05.450969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '396ec2cf84a34efe9cc84b30614db294', 'request_id': 'req-d5113621-c6be-4198-b465-9c31f05a29e1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:05.461184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '396ec2cf84a34efe9cc84b30614db294', 'request_id': 'req-d5113621-c6be-4198-b465-9c31f05a29e1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:05.487838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] The token cache is being invalidate because project 396ec2cf84a34efe9cc84b30614db294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:05.488169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:05.488881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:05.490603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 927/1831] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:05 2022] DELETE /identity/v3/projects/396ec2cf84a34efe9cc84b30614db294 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:05.495565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc57c03-c757-46b2-bdc7-80f67a7aac1d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.495955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc57c03-c757-46b2-bdc7-80f67a7aac1d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.496225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc57c03-c757-46b2-bdc7-80f67a7aac1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.507606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:05.509383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:05.520146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8cc57c03-c757-46b2-bdc7-80f67a7aac1d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:05.522244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8cc57c03-c757-46b2-bdc7-80f67a7aac1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:05.590446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5113621-c6be-4198-b465-9c31f05a29e1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:05.595983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c4cac-f03f-424c-a9dc-f0a80b1f5283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:05.596781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c4cac-f03f-424c-a9dc-f0a80b1f5283 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.597057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c4cac-f03f-424c-a9dc-f0a80b1f5283 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.597331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c4cac-f03f-424c-a9dc-f0a80b1f5283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.599099 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c19c4cac-f03f-424c-a9dc-f0a80b1f5283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:05.612872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 905/1832] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:05.622315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f3197-613c-41e5-93ef-405b4d1393d9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.623644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f3197-613c-41e5-93ef-405b4d1393d9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.624018 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f3197-613c-41e5-93ef-405b4d1393d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.653027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-378f3197-613c-41e5-93ef-405b4d1393d9 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:05.655047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-378f3197-613c-41e5-93ef-405b4d1393d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:05.663886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 906/1833] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:05.668064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f3197-613c-41e5-93ef-405b4d1393d9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:05.672406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c4cac-f03f-424c-a9dc-f0a80b1f5283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:05.673863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c4cac-f03f-424c-a9dc-f0a80b1f5283 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:05.677941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f3197-613c-41e5-93ef-405b4d1393d9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:05.684079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 928/1834] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:05 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:21:05.684663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236a8ee-fb9f-4ba8-b27b-b2aebccee2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:05.685935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236a8ee-fb9f-4ba8-b27b-b2aebccee2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.686128 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236a8ee-fb9f-4ba8-b27b-b2aebccee2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.686395 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236a8ee-fb9f-4ba8-b27b-b2aebccee2d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.687043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236a8ee-fb9f-4ba8-b27b-b2aebccee2d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:21:05.687344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236a8ee-fb9f-4ba8-b27b-b2aebccee2d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:05.689248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236a8ee-fb9f-4ba8-b27b-b2aebccee2d1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:05.706374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 907/1835] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:21:05.711186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c4cac-f03f-424c-a9dc-f0a80b1f5283 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:05.723750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c4cac-f03f-424c-a9dc-f0a80b1f5283 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:05.728964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba431f90-8fe8-4f8f-ab44-81e5cb854481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:05.729586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba431f90-8fe8-4f8f-ab44-81e5cb854481 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.729869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba431f90-8fe8-4f8f-ab44-81e5cb854481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.730251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba431f90-8fe8-4f8f-ab44-81e5cb854481 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.731027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba431f90-8fe8-4f8f-ab44-81e5cb854481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:05.734002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba431f90-8fe8-4f8f-ab44-81e5cb854481 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:05.785073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 929/1836] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:05 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:05.791005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236a8ee-fb9f-4ba8-b27b-b2aebccee2d1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:05.803160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236a8ee-fb9f-4ba8-b27b-b2aebccee2d1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:05.921197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567b603-477a-495a-96db-8d8d80e42c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:05.922747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b603-477a-495a-96db-8d8d80e42c0d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:05.923261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b603-477a-495a-96db-8d8d80e42c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:05.923783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b603-477a-495a-96db-8d8d80e42c0d admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:05.925331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567b603-477a-495a-96db-8d8d80e42c0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:05.929635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567b603-477a-495a-96db-8d8d80e42c0d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:05.970641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 908/1837] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:05 2022] POST /identity/v3/users => generated 294 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:05.974829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba431f90-8fe8-4f8f-ab44-81e5cb854481 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:05.990871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba431f90-8fe8-4f8f-ab44-81e5cb854481 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.005864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406f759-6dff-4f16-b3e4-e4ac6f8dcc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.006991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f759-6dff-4f16-b3e4-e4ac6f8dcc6e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.007468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f759-6dff-4f16-b3e4-e4ac6f8dcc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.007842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406f759-6dff-4f16-b3e4-e4ac6f8dcc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.008446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406f759-6dff-4f16-b3e4-e4ac6f8dcc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:06.010158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c406f759-6dff-4f16-b3e4-e4ac6f8dcc6e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:06.055807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 930/1838] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:06.058989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567b603-477a-495a-96db-8d8d80e42c0d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:06.086386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567b603-477a-495a-96db-8d8d80e42c0d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.099737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd3771-55d0-45d5-b088-19848d97b3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.100380 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd3771-55d0-45d5-b088-19848d97b3db admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.100792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd3771-55d0-45d5-b088-19848d97b3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.100911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd3771-55d0-45d5-b088-19848d97b3db admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.101566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd3771-55d0-45d5-b088-19848d97b3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:06.103460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bd3771-55d0-45d5-b088-19848d97b3db admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:06.111026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:06.125108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 909/1839] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:06.127464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406f759-6dff-4f16-b3e4-e4ac6f8dcc6e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:06.137989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406f759-6dff-4f16-b3e4-e4ac6f8dcc6e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.145412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388052e-6412-4852-b131-8b664b5c9235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.146464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388052e-6412-4852-b131-8b664b5c9235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.147064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388052e-6412-4852-b131-8b664b5c9235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.148007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388052e-6412-4852-b131-8b664b5c9235 admin admin] PATH_INFO: `/v3/projects/843e106fe5574a968e010b0c8e5a6b5e/users/6177d0330c5f4e4aa495be06ba5a5a19/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.186459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388052e-6412-4852-b131-8b664b5c9235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=843e106fe5574a968e010b0c8e5a6b5e, user_id=6177d0330c5f4e4aa495be06ba5a5a19, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:06.190939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388052e-6412-4852-b131-8b664b5c9235 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:06.202115 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:06.227805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 931/1840] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:06 2022] PUT /identity/v3/projects/843e106fe5574a968e010b0c8e5a6b5e/users/6177d0330c5f4e4aa495be06ba5a5a19/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:06.235946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1bef1-2182-45f5-b7e5-d0c8cc18544a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.236227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1bef1-2182-45f5-b7e5-d0c8cc18544a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.236474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1bef1-2182-45f5-b7e5-d0c8cc18544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.277475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e1b1bef1-2182-45f5-b7e5-d0c8cc18544a None None] MFA Rules not processed for user `6177d0330c5f4e4aa495be06ba5a5a19`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:06.279640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e1b1bef1-2182-45f5-b7e5-d0c8cc18544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:06.364484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 910/1841] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:06.368944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388052e-6412-4852-b131-8b664b5c9235 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:06.437146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388052e-6412-4852-b131-8b664b5c9235 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.444254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d050d0-115c-430d-8753-00db4e29b210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.445032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d050d0-115c-430d-8753-00db4e29b210 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.445453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d050d0-115c-430d-8753-00db4e29b210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.445816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d050d0-115c-430d-8753-00db4e29b210 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.446466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d050d0-115c-430d-8753-00db4e29b210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:06.448412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d050d0-115c-430d-8753-00db4e29b210 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:06.481083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 932/1842] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:06 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:21:06.485277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1bef1-2182-45f5-b7e5-d0c8cc18544a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:06.495848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1bef1-2182-45f5-b7e5-d0c8cc18544a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.551944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ff4c9-3911-48c1-800c-17105307701a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.552473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ff4c9-3911-48c1-800c-17105307701a admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.553038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ff4c9-3911-48c1-800c-17105307701a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.553316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ff4c9-3911-48c1-800c-17105307701a admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.554057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ff4c9-3911-48c1-800c-17105307701a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:06.556483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ff4c9-3911-48c1-800c-17105307701a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:06.582225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 911/1843] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:06 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:06.583654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d050d0-115c-430d-8753-00db4e29b210 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:06.593907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d050d0-115c-430d-8753-00db4e29b210 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.599908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648980ce-2fd1-415d-a630-2d70506573db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.600952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648980ce-2fd1-415d-a630-2d70506573db admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.601464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648980ce-2fd1-415d-a630-2d70506573db admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.601914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648980ce-2fd1-415d-a630-2d70506573db admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.602563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648980ce-2fd1-415d-a630-2d70506573db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:06.604210 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648980ce-2fd1-415d-a630-2d70506573db admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:06.616035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 933/1844] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:21:06.620404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ff4c9-3911-48c1-800c-17105307701a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:06.631368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ff4c9-3911-48c1-800c-17105307701a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.636032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cc85e-5d2c-48e9-bc47-5e0655cdf42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.636784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cc85e-5d2c-48e9-bc47-5e0655cdf42a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.637067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cc85e-5d2c-48e9-bc47-5e0655cdf42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.637341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cc85e-5d2c-48e9-bc47-5e0655cdf42a admin admin] PATH_INFO: `/v3/projects/bd980ccee39541668339b7a0f569ba4e/users/e04cdf5c39c24e95bd555b4a5f7c36ac/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.652589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cc85e-5d2c-48e9-bc47-5e0655cdf42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd980ccee39541668339b7a0f569ba4e, user_id=e04cdf5c39c24e95bd555b4a5f7c36ac, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:06.655302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cc85e-5d2c-48e9-bc47-5e0655cdf42a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:06.668400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 912/1845] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:06 2022] PUT /identity/v3/projects/bd980ccee39541668339b7a0f569ba4e/users/e04cdf5c39c24e95bd555b4a5f7c36ac/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:06.672913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648980ce-2fd1-415d-a630-2d70506573db admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:06.682855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648980ce-2fd1-415d-a630-2d70506573db admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.738845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdff9ec-f3e3-4ab5-824b-cd41577dca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.739455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff9ec-f3e3-4ab5-824b-cd41577dca98 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.739737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff9ec-f3e3-4ab5-824b-cd41577dca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.740047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff9ec-f3e3-4ab5-824b-cd41577dca98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.740680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdff9ec-f3e3-4ab5-824b-cd41577dca98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:06.742318 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdff9ec-f3e3-4ab5-824b-cd41577dca98 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:06.753241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 934/1846] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:06.757595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cc85e-5d2c-48e9-bc47-5e0655cdf42a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:06.768457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cc85e-5d2c-48e9-bc47-5e0655cdf42a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.773016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82750308-d98e-4619-9064-fbc2ae90ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.773668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82750308-d98e-4619-9064-fbc2ae90ce40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.774032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82750308-d98e-4619-9064-fbc2ae90ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.774302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82750308-d98e-4619-9064-fbc2ae90ce40 admin admin] PATH_INFO: `/v3/projects/bd980ccee39541668339b7a0f569ba4e/users/e04cdf5c39c24e95bd555b4a5f7c36ac/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.776203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82750308-d98e-4619-9064-fbc2ae90ce40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd980ccee39541668339b7a0f569ba4e, user_id=e04cdf5c39c24e95bd555b4a5f7c36ac, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:06.778823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82750308-d98e-4619-9064-fbc2ae90ce40 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:06.789373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 913/1847] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:06 2022] PUT /identity/v3/projects/bd980ccee39541668339b7a0f569ba4e/users/e04cdf5c39c24e95bd555b4a5f7c36ac/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:06.800222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c3b1f-a8fe-4500-953b-d60a97658082 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.800508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c3b1f-a8fe-4500-953b-d60a97658082 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.800816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c3b1f-a8fe-4500-953b-d60a97658082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.828488 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e58c3b1f-a8fe-4500-953b-d60a97658082 None None] MFA Rules not processed for user `e04cdf5c39c24e95bd555b4a5f7c36ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:06.829698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e58c3b1f-a8fe-4500-953b-d60a97658082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:06.886569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:06.907028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 935/1848] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:06.918675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82750308-d98e-4619-9064-fbc2ae90ce40 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:06.968624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:06.984947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82750308-d98e-4619-9064-fbc2ae90ce40 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:06.988600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cca1f1-1f8d-42ba-8219-a7c463885447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:06.989869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cca1f1-1f8d-42ba-8219-a7c463885447 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:06.989869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cca1f1-1f8d-42ba-8219-a7c463885447 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:06.990049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cca1f1-1f8d-42ba-8219-a7c463885447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:06.999673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cca1f1-1f8d-42ba-8219-a7c463885447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:07.001130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cca1f1-1f8d-42ba-8219-a7c463885447 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:07.011121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 914/1849] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:07.040979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c3b1f-a8fe-4500-953b-d60a97658082 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:07.106663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c3b1f-a8fe-4500-953b-d60a97658082 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:07.112029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:07.112692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:07.112974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:07.113251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] PATH_INFO: `/v3/users/6177d0330c5f4e4aa495be06ba5a5a19` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:07.114432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6177d0330c5f4e4aa495be06ba5a5a19)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:07.116425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:07.173630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6177d0330c5f4e4aa495be06ba5a5a19'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.181050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6177d0330c5f4e4aa495be06ba5a5a19'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.187922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6177d0330c5f4e4aa495be06ba5a5a19'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.187922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6177d0330c5f4e4aa495be06ba5a5a19'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.190457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6177d0330c5f4e4aa495be06ba5a5a19'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.215174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 936/1850] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:07 2022] DELETE /identity/v3/users/6177d0330c5f4e4aa495be06ba5a5a19 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:07.220315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ba49-51e1-40dc-a5dc-456aca860b33 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:07.220605 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ba49-51e1-40dc-a5dc-456aca860b33 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:07.220877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629ba49-51e1-40dc-a5dc-456aca860b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:07.243683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7629ba49-51e1-40dc-a5dc-456aca860b33 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:07.245324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7629ba49-51e1-40dc-a5dc-456aca860b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:07.312195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 915/1851] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:07.319579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:07.321124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:07.381136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2c1bc-1155-44dd-a9bd-3053f17c51fd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:07.384934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e6195-f222-4d38-84f7-86a73d634d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:07.385227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e6195-f222-4d38-84f7-86a73d634d57 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:07.385508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e6195-f222-4d38-84f7-86a73d634d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:07.385774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e6195-f222-4d38-84f7-86a73d634d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:07.395496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e6195-f222-4d38-84f7-86a73d634d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:07.396960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e6195-f222-4d38-84f7-86a73d634d57 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:07.406895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 937/1852] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:07.681670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ba49-51e1-40dc-a5dc-456aca860b33 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:07.683299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7629ba49-51e1-40dc-a5dc-456aca860b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:07.695860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629ba49-51e1-40dc-a5dc-456aca860b33 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:07.701922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:07.702578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:07.702868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:07.703137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] PATH_INFO: `/v3/projects/843e106fe5574a968e010b0c8e5a6b5e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:07.704330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=843e106fe5574a968e010b0c8e5a6b5e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:07.706271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:07.753687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '843e106fe5574a968e010b0c8e5a6b5e', 'request_id': 'req-8e1d7520-132e-4ed8-bfd0-d372ef20a567'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.760106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '843e106fe5574a968e010b0c8e5a6b5e', 'request_id': 'req-8e1d7520-132e-4ed8-bfd0-d372ef20a567'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.768259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '843e106fe5574a968e010b0c8e5a6b5e', 'request_id': 'req-8e1d7520-132e-4ed8-bfd0-d372ef20a567'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.778666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] The token cache is being invalidate because project 843e106fe5574a968e010b0c8e5a6b5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:07.778978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.779612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.781161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 916/1853] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:07 2022] DELETE /identity/v3/projects/843e106fe5574a968e010b0c8e5a6b5e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:07.785122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e6195-f222-4d38-84f7-86a73d634d57 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:07.794767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e6195-f222-4d38-84f7-86a73d634d57 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:07.803468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:07.854280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:07.854918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:07.855200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:07.855472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] PATH_INFO: `/v3/users/e04cdf5c39c24e95bd555b4a5f7c36ac` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:07.856637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e04cdf5c39c24e95bd555b4a5f7c36ac)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:07.858591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:07.914918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e04cdf5c39c24e95bd555b4a5f7c36ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.922701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e04cdf5c39c24e95bd555b4a5f7c36ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.923444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e04cdf5c39c24e95bd555b4a5f7c36ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.924146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e04cdf5c39c24e95bd555b4a5f7c36ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.931113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8efce01f-b01a-4279-8019-21b6fc4d23c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e04cdf5c39c24e95bd555b4a5f7c36ac'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:07.948991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 938/1854] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:07 2022] DELETE /identity/v3/users/e04cdf5c39c24e95bd555b4a5f7c36ac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:08.425491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:08.426904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:08.441233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d7520-132e-4ed8-bfd0-d372ef20a567 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:08.502378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:08.503323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:08.503603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:08.503895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] PATH_INFO: `/v3/projects/bd980ccee39541668339b7a0f569ba4e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:08.505036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd980ccee39541668339b7a0f569ba4e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:08.506913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:08.514859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:08.559694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd980ccee39541668339b7a0f569ba4e', 'request_id': 'req-f3e0c458-ac78-4a99-b72d-9abff67d848e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:08.566921 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd980ccee39541668339b7a0f569ba4e', 'request_id': 'req-f3e0c458-ac78-4a99-b72d-9abff67d848e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:08.574666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd980ccee39541668339b7a0f569ba4e', 'request_id': 'req-f3e0c458-ac78-4a99-b72d-9abff67d848e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:08.586243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] The token cache is being invalidate because project bd980ccee39541668339b7a0f569ba4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:08.586529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:08.587362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f3e0c458-ac78-4a99-b72d-9abff67d848e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:08.589505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 917/1855] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:08 2022] DELETE /identity/v3/projects/bd980ccee39541668339b7a0f569ba4e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:08.594780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af482-1563-4a57-8176-cd77cc02ffae None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:08.595106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af482-1563-4a57-8176-cd77cc02ffae None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:08.595397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af482-1563-4a57-8176-cd77cc02ffae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:08.615731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b73af482-1563-4a57-8176-cd77cc02ffae None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:08.616957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b73af482-1563-4a57-8176-cd77cc02ffae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:08.674192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 939/1856] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:08.681920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d6ab4-974f-4532-95bb-951324915cf2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:08.682147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d6ab4-974f-4532-95bb-951324915cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:08.682423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d6ab4-974f-4532-95bb-951324915cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:08.702454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c78d6ab4-974f-4532-95bb-951324915cf2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:08.703668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c78d6ab4-974f-4532-95bb-951324915cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:08.712199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 918/1857] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:08.717065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73af482-1563-4a57-8176-cd77cc02ffae None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:08.726800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73af482-1563-4a57-8176-cd77cc02ffae None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:08.731468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7dd66-c0e8-4a53-a02d-1f1b80d20513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:08.732694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7dd66-c0e8-4a53-a02d-1f1b80d20513 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:08.732989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7dd66-c0e8-4a53-a02d-1f1b80d20513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:08.733260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7dd66-c0e8-4a53-a02d-1f1b80d20513 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:08.733918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7dd66-c0e8-4a53-a02d-1f1b80d20513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:21:08.734225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7dd66-c0e8-4a53-a02d-1f1b80d20513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:08.735887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7dd66-c0e8-4a53-a02d-1f1b80d20513 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:08.749545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 940/1858] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:21:08.753922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d6ab4-974f-4532-95bb-951324915cf2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:08.763613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d6ab4-974f-4532-95bb-951324915cf2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:08.768424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5207e-f9ab-44bf-8736-eaa719a40adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:08.769036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5207e-f9ab-44bf-8736-eaa719a40adb admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:08.769319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5207e-f9ab-44bf-8736-eaa719a40adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:08.769616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5207e-f9ab-44bf-8736-eaa719a40adb admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:08.770245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5207e-f9ab-44bf-8736-eaa719a40adb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:08.772215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5207e-f9ab-44bf-8736-eaa719a40adb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:08.803702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 919/1859] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:08 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:08.804897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7dd66-c0e8-4a53-a02d-1f1b80d20513 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:08.814832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7dd66-c0e8-4a53-a02d-1f1b80d20513 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:08.831890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:08.872958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbc843-c59e-4a82-a1c7-ee136de78bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:08.873699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbc843-c59e-4a82-a1c7-ee136de78bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:08.873993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbc843-c59e-4a82-a1c7-ee136de78bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:08.874301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbc843-c59e-4a82-a1c7-ee136de78bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:08.874933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbc843-c59e-4a82-a1c7-ee136de78bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:08.876932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbc843-c59e-4a82-a1c7-ee136de78bd1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:08.898489 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 941/1860] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:08 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:21:08.902698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5207e-f9ab-44bf-8736-eaa719a40adb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:08.913429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5207e-f9ab-44bf-8736-eaa719a40adb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:08.917978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f7054-3d16-4d3f-9256-409abcbd997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:08.918588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7054-3d16-4d3f-9256-409abcbd997e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:08.918869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7054-3d16-4d3f-9256-409abcbd997e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:08.919145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f7054-3d16-4d3f-9256-409abcbd997e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:08.919955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f7054-3d16-4d3f-9256-409abcbd997e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:08.921561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f7054-3d16-4d3f-9256-409abcbd997e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:08.932029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 920/1861] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:08.936033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbc843-c59e-4a82-a1c7-ee136de78bd1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:08.946221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbc843-c59e-4a82-a1c7-ee136de78bd1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:08.951291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05fef7-cefb-4a0e-9940-a279ff902238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:08.951924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05fef7-cefb-4a0e-9940-a279ff902238 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:08.952186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05fef7-cefb-4a0e-9940-a279ff902238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:08.952453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05fef7-cefb-4a0e-9940-a279ff902238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:08.953086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05fef7-cefb-4a0e-9940-a279ff902238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:08.954658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05fef7-cefb-4a0e-9940-a279ff902238 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:08.965234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 942/1862] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:08.970427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f7054-3d16-4d3f-9256-409abcbd997e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:08.980517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f7054-3d16-4d3f-9256-409abcbd997e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:08.985251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897ecbc-8c07-4434-a16e-1a16b78d0741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:08.985909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897ecbc-8c07-4434-a16e-1a16b78d0741 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:08.986186 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897ecbc-8c07-4434-a16e-1a16b78d0741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:08.986461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897ecbc-8c07-4434-a16e-1a16b78d0741 admin admin] PATH_INFO: `/v3/projects/9837dfd6f04f48e0b1f17cdd0b00e6cb/users/133c308cc3ed42209f3ed8e2c47f6488/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.001097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897ecbc-8c07-4434-a16e-1a16b78d0741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9837dfd6f04f48e0b1f17cdd0b00e6cb, user_id=133c308cc3ed42209f3ed8e2c47f6488, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:09.003812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897ecbc-8c07-4434-a16e-1a16b78d0741 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:09.016846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 921/1863] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:08 2022] PUT /identity/v3/projects/9837dfd6f04f48e0b1f17cdd0b00e6cb/users/133c308cc3ed42209f3ed8e2c47f6488/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:09.028216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c89670-224b-450c-877d-582c15bf826a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:09.028498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c89670-224b-450c-877d-582c15bf826a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:09.029380 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c89670-224b-450c-877d-582c15bf826a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.057002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-19c89670-224b-450c-877d-582c15bf826a None None] MFA Rules not processed for user `133c308cc3ed42209f3ed8e2c47f6488`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:09.058213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-19c89670-224b-450c-877d-582c15bf826a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:09.125353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 943/1864] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:09.129814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897ecbc-8c07-4434-a16e-1a16b78d0741 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:09.193087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897ecbc-8c07-4434-a16e-1a16b78d0741 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:09.197820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fc054-90bb-4995-8cc2-196d00f46b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:09.198440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fc054-90bb-4995-8cc2-196d00f46b44 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:09.198718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fc054-90bb-4995-8cc2-196d00f46b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:09.198993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fc054-90bb-4995-8cc2-196d00f46b44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.199639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fc054-90bb-4995-8cc2-196d00f46b44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:09.202202 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fc054-90bb-4995-8cc2-196d00f46b44 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:09.232504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 922/1865] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:09 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:21:09.233891 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c89670-224b-450c-877d-582c15bf826a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:09.244581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c89670-224b-450c-877d-582c15bf826a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:09.320710 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444bde0-d541-4b89-8f2a-5083bde5ddb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:09.321448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444bde0-d541-4b89-8f2a-5083bde5ddb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:09.322301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444bde0-d541-4b89-8f2a-5083bde5ddb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:09.322613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444bde0-d541-4b89-8f2a-5083bde5ddb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.323330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444bde0-d541-4b89-8f2a-5083bde5ddb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:09.325465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444bde0-d541-4b89-8f2a-5083bde5ddb1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:09.355844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 944/1866] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:09 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:09.357261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fc054-90bb-4995-8cc2-196d00f46b44 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:09.371313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fc054-90bb-4995-8cc2-196d00f46b44 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:09.386154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436c1e8-788b-4e02-a8cc-3c7c65e00862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:09.387666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436c1e8-788b-4e02-a8cc-3c7c65e00862 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:09.388193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436c1e8-788b-4e02-a8cc-3c7c65e00862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:09.389293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436c1e8-788b-4e02-a8cc-3c7c65e00862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.390128 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436c1e8-788b-4e02-a8cc-3c7c65e00862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:09.392653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436c1e8-788b-4e02-a8cc-3c7c65e00862 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:09.407345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 923/1867] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:09.409927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444bde0-d541-4b89-8f2a-5083bde5ddb1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:09.420529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444bde0-d541-4b89-8f2a-5083bde5ddb1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:09.428081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fd1aa-a423-4224-aa84-ac334545ca9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:09.428877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fd1aa-a423-4224-aa84-ac334545ca9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:09.429292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fd1aa-a423-4224-aa84-ac334545ca9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:09.429642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0fd1aa-a423-4224-aa84-ac334545ca9e admin admin] PATH_INFO: `/v3/projects/15b32b5de0c542e4b4da5e8f3e9d19c7/users/9e2c0d651752456493e7e0a62bb7705a/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.455071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fd1aa-a423-4224-aa84-ac334545ca9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b32b5de0c542e4b4da5e8f3e9d19c7, user_id=9e2c0d651752456493e7e0a62bb7705a, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:09.459463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0fd1aa-a423-4224-aa84-ac334545ca9e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:09.463152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:09.484912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 945/1868] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:09 2022] PUT /identity/v3/projects/15b32b5de0c542e4b4da5e8f3e9d19c7/users/9e2c0d651752456493e7e0a62bb7705a/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:09.486668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436c1e8-788b-4e02-a8cc-3c7c65e00862 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:09.504593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436c1e8-788b-4e02-a8cc-3c7c65e00862 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:09.512208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:09.591310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfa124-5c41-4fee-bccd-38daa090650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:09.591959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa124-5c41-4fee-bccd-38daa090650c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:09.592300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa124-5c41-4fee-bccd-38daa090650c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:09.592571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfa124-5c41-4fee-bccd-38daa090650c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.593225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfa124-5c41-4fee-bccd-38daa090650c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:09.594901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfa124-5c41-4fee-bccd-38daa090650c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:09.605851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 924/1869] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:09.611642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fd1aa-a423-4224-aa84-ac334545ca9e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:09.623203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0fd1aa-a423-4224-aa84-ac334545ca9e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:09.628028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39907068-b042-404a-9452-2d9f068dd53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:09.628685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39907068-b042-404a-9452-2d9f068dd53a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:09.628961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39907068-b042-404a-9452-2d9f068dd53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:09.629231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39907068-b042-404a-9452-2d9f068dd53a admin admin] PATH_INFO: `/v3/projects/15b32b5de0c542e4b4da5e8f3e9d19c7/users/9e2c0d651752456493e7e0a62bb7705a/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.631129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39907068-b042-404a-9452-2d9f068dd53a admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b32b5de0c542e4b4da5e8f3e9d19c7, user_id=9e2c0d651752456493e7e0a62bb7705a, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:09.634269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39907068-b042-404a-9452-2d9f068dd53a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:09.646310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 946/1870] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:09 2022] PUT /identity/v3/projects/15b32b5de0c542e4b4da5e8f3e9d19c7/users/9e2c0d651752456493e7e0a62bb7705a/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:09.656912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18224-01a2-4040-b85e-74c9e3024609 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:09.657192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18224-01a2-4040-b85e-74c9e3024609 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:09.657482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18224-01a2-4040-b85e-74c9e3024609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.699386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-afc18224-01a2-4040-b85e-74c9e3024609 None None] MFA Rules not processed for user `9e2c0d651752456493e7e0a62bb7705a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:09.700595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-afc18224-01a2-4040-b85e-74c9e3024609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:09.777251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 925/1871] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:09 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:09.788447 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39907068-b042-404a-9452-2d9f068dd53a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:09.790169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-39907068-b042-404a-9452-2d9f068dd53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:09.851985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39907068-b042-404a-9452-2d9f068dd53a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:09.855919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86150b0-2f76-470a-bc5b-6e26c7d8cb65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:09.857368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86150b0-2f76-470a-bc5b-6e26c7d8cb65 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:09.857368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86150b0-2f76-470a-bc5b-6e26c7d8cb65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:09.857615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86150b0-2f76-470a-bc5b-6e26c7d8cb65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:09.867096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86150b0-2f76-470a-bc5b-6e26c7d8cb65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:09.868583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86150b0-2f76-470a-bc5b-6e26c7d8cb65 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:09.880034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 947/1872] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:10.019090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18224-01a2-4040-b85e-74c9e3024609 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:10.081237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18224-01a2-4040-b85e-74c9e3024609 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:10.086038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:10.086672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:10.086977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:10.087250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] PATH_INFO: `/v3/users/133c308cc3ed42209f3ed8e2c47f6488` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:10.088377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=133c308cc3ed42209f3ed8e2c47f6488)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:10.090287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:10.124795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:10.139217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133c308cc3ed42209f3ed8e2c47f6488'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:10.139923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133c308cc3ed42209f3ed8e2c47f6488'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:10.147018 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133c308cc3ed42209f3ed8e2c47f6488'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:10.147633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133c308cc3ed42209f3ed8e2c47f6488'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:10.154738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133c308cc3ed42209f3ed8e2c47f6488'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:10.174853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 926/1873] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:10 2022] DELETE /identity/v3/users/133c308cc3ed42209f3ed8e2c47f6488 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:10.179441 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24346ad-1f26-4e01-9f51-ecfc8e57e77c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:10.179730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24346ad-1f26-4e01-9f51-ecfc8e57e77c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:10.180035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24346ad-1f26-4e01-9f51-ecfc8e57e77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:10.201690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d24346ad-1f26-4e01-9f51-ecfc8e57e77c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:10.203193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d24346ad-1f26-4e01-9f51-ecfc8e57e77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:10.283192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 948/1874] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:10.293659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:10.295166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:10.383007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba56984-cfb1-418b-92a6-9d4d662b1fbd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:10.397937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5494ec-ca49-4898-9fdb-3b05084cf15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:10.399251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5494ec-ca49-4898-9fdb-3b05084cf15f service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:10.399564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5494ec-ca49-4898-9fdb-3b05084cf15f service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:10.400740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5494ec-ca49-4898-9fdb-3b05084cf15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:10.440663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5494ec-ca49-4898-9fdb-3b05084cf15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:10.442685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5494ec-ca49-4898-9fdb-3b05084cf15f service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:10.460050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 927/1875] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:10.918346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24346ad-1f26-4e01-9f51-ecfc8e57e77c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:10.921991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d24346ad-1f26-4e01-9f51-ecfc8e57e77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:10.938878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24346ad-1f26-4e01-9f51-ecfc8e57e77c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:10.947636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:10.948557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:10.948966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:10.949393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] PATH_INFO: `/v3/projects/9837dfd6f04f48e0b1f17cdd0b00e6cb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:10.964665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9837dfd6f04f48e0b1f17cdd0b00e6cb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:10.967359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:11.041139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9837dfd6f04f48e0b1f17cdd0b00e6cb', 'request_id': 'req-7a40abf8-5b58-420b-aff5-815b4c79f590'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.049134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9837dfd6f04f48e0b1f17cdd0b00e6cb', 'request_id': 'req-7a40abf8-5b58-420b-aff5-815b4c79f590'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.058778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9837dfd6f04f48e0b1f17cdd0b00e6cb', 'request_id': 'req-7a40abf8-5b58-420b-aff5-815b4c79f590'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.076421 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] The token cache is being invalidate because project 9837dfd6f04f48e0b1f17cdd0b00e6cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:11.076708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.077984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a40abf8-5b58-420b-aff5-815b4c79f590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.082533 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 949/1876] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:10 2022] DELETE /identity/v3/projects/9837dfd6f04f48e0b1f17cdd0b00e6cb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:11.085517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5494ec-ca49-4898-9fdb-3b05084cf15f service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:11.098750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5494ec-ca49-4898-9fdb-3b05084cf15f service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:11.173319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c4b1d-1c8e-4516-8aca-5f1cb313cd83 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:11.173804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c4b1d-1c8e-4516-8aca-5f1cb313cd83 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:11.174225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c4b1d-1c8e-4516-8aca-5f1cb313cd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:11.199884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:11.284752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:11.286783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:11.291322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:11.292441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] PATH_INFO: `/v3/users/9e2c0d651752456493e7e0a62bb7705a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:11.297259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e2c0d651752456493e7e0a62bb7705a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:11.300881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:11.412980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b49c4b1d-1c8e-4516-8aca-5f1cb313cd83 None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:11.422809 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b49c4b1d-1c8e-4516-8aca-5f1cb313cd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:11.435700 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e2c0d651752456493e7e0a62bb7705a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.436978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e2c0d651752456493e7e0a62bb7705a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.464102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:11.484035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e2c0d651752456493e7e0a62bb7705a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.489107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e2c0d651752456493e7e0a62bb7705a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.527288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e2c0d651752456493e7e0a62bb7705a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:11.580416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 928/1877] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:11 2022] DELETE /identity/v3/users/9e2c0d651752456493e7e0a62bb7705a => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:11.590343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 950/1878] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:21:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:11.732651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:11.734766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:12.005336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680b483-ea18-4fac-9982-aec214981428 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:12.012192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c462225-1637-4b46-a226-d86086a11825 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:12.013304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c462225-1637-4b46-a226-d86086a11825 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:12.013304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c462225-1637-4b46-a226-d86086a11825 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:12.013493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c462225-1637-4b46-a226-d86086a11825 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:12.211643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c462225-1637-4b46-a226-d86086a11825 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:12.217214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c462225-1637-4b46-a226-d86086a11825 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:12.250727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 929/1879] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:21:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:21:12.600151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c4b1d-1c8e-4516-8aca-5f1cb313cd83 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:12.603586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b49c4b1d-1c8e-4516-8aca-5f1cb313cd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:12.625441 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c4b1d-1c8e-4516-8aca-5f1cb313cd83 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:12.742813 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:12.743487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:12.743787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:12.744075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] PATH_INFO: `/v3/projects/15b32b5de0c542e4b4da5e8f3e9d19c7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:12.745491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b32b5de0c542e4b4da5e8f3e9d19c7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:12.747466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:12.823067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b32b5de0c542e4b4da5e8f3e9d19c7', 'request_id': 'req-f70927ae-5d72-495e-b2be-eb59bf134d47'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:12.851332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b32b5de0c542e4b4da5e8f3e9d19c7', 'request_id': 'req-f70927ae-5d72-495e-b2be-eb59bf134d47'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:12.875577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b32b5de0c542e4b4da5e8f3e9d19c7', 'request_id': 'req-f70927ae-5d72-495e-b2be-eb59bf134d47'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:12.924870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] The token cache is being invalidate because project 15b32b5de0c542e4b4da5e8f3e9d19c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:12.925565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:12.926095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f70927ae-5d72-495e-b2be-eb59bf134d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:12.939721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 951/1880] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:12 2022] DELETE /identity/v3/projects/15b32b5de0c542e4b4da5e8f3e9d19c7 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:12.941028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c0080-cc62-4fbf-abc3-4b2a69f04432 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:12.941376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c0080-cc62-4fbf-abc3-4b2a69f04432 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:12.941659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c0080-cc62-4fbf-abc3-4b2a69f04432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.000309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4a0c0080-cc62-4fbf-abc3-4b2a69f04432 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:13.006457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4a0c0080-cc62-4fbf-abc3-4b2a69f04432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:13.102389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:13.130410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 930/1881] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:13.138635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a3edc-69fb-4db0-b9f4-2fd3db769619 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.139196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a3edc-69fb-4db0-b9f4-2fd3db769619 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.140009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a3edc-69fb-4db0-b9f4-2fd3db769619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.174831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3c3a3edc-69fb-4db0-b9f4-2fd3db769619 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:13.178398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3c3a3edc-69fb-4db0-b9f4-2fd3db769619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:13.218934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c0080-cc62-4fbf-abc3-4b2a69f04432 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:13.222522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 952/1882] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:13.235277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c0080-cc62-4fbf-abc3-4b2a69f04432 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:13.241525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2b25-8841-4333-8c67-bf3c7c71c414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:13.242224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2b25-8841-4333-8c67-bf3c7c71c414 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.242448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2b25-8841-4333-8c67-bf3c7c71c414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.242734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2b25-8841-4333-8c67-bf3c7c71c414 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.244621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa2b25-8841-4333-8c67-bf3c7c71c414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:21:13.245097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa2b25-8841-4333-8c67-bf3c7c71c414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:13.248079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa2b25-8841-4333-8c67-bf3c7c71c414 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:13.272307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 931/1883] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:21:13.276391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a3edc-69fb-4db0-b9f4-2fd3db769619 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:13.283012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:13.288341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a3edc-69fb-4db0-b9f4-2fd3db769619 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:13.325696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3cacc-9de2-4315-acb9-ab4acce3f481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:13.326367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3cacc-9de2-4315-acb9-ab4acce3f481 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.326635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3cacc-9de2-4315-acb9-ab4acce3f481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.327418 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3cacc-9de2-4315-acb9-ab4acce3f481 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.328137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3cacc-9de2-4315-acb9-ab4acce3f481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:13.330574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3cacc-9de2-4315-acb9-ab4acce3f481 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:13.395106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2b25-8841-4333-8c67-bf3c7c71c414 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:13.396432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 953/1884] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:13 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:13.415528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2b25-8841-4333-8c67-bf3c7c71c414 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:13.489762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1228e0-cb8c-4d23-80f3-adeb5aafe8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:13.490567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1228e0-cb8c-4d23-80f3-adeb5aafe8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.490679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1228e0-cb8c-4d23-80f3-adeb5aafe8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.490910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1228e0-cb8c-4d23-80f3-adeb5aafe8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.491555 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1228e0-cb8c-4d23-80f3-adeb5aafe8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:13.493514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1228e0-cb8c-4d23-80f3-adeb5aafe8f0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:13.519956 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 932/1885] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:13 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:21:13.520230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3cacc-9de2-4315-acb9-ab4acce3f481 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:13.530621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3cacc-9de2-4315-acb9-ab4acce3f481 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:13.535260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed0747-c0a7-4d8d-9cea-0f78c317677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:13.535896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed0747-c0a7-4d8d-9cea-0f78c317677b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.536179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed0747-c0a7-4d8d-9cea-0f78c317677b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.536446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed0747-c0a7-4d8d-9cea-0f78c317677b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.537084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed0747-c0a7-4d8d-9cea-0f78c317677b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:13.538670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed0747-c0a7-4d8d-9cea-0f78c317677b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:13.549340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 954/1886] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:21:13.553392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1228e0-cb8c-4d23-80f3-adeb5aafe8f0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:13.564199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1228e0-cb8c-4d23-80f3-adeb5aafe8f0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:13.568932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48507c72-590a-4252-8c1f-cee823335fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:13.569539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48507c72-590a-4252-8c1f-cee823335fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.569948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48507c72-590a-4252-8c1f-cee823335fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.570252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48507c72-590a-4252-8c1f-cee823335fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.570927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48507c72-590a-4252-8c1f-cee823335fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:13.572568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48507c72-590a-4252-8c1f-cee823335fc2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:13.585095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 933/1887] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:13.589485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed0747-c0a7-4d8d-9cea-0f78c317677b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:13.599584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed0747-c0a7-4d8d-9cea-0f78c317677b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:13.605191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe81e06-4504-4828-b597-5af86ec85b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:13.605956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe81e06-4504-4828-b597-5af86ec85b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.606304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe81e06-4504-4828-b597-5af86ec85b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.606647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe81e06-4504-4828-b597-5af86ec85b9c admin admin] PATH_INFO: `/v3/projects/86f711618beb4eaa90b4b38bc938bbdd/users/92b4a96c11dc49fd8ae2145fb6fef61e/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.622443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe81e06-4504-4828-b597-5af86ec85b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f711618beb4eaa90b4b38bc938bbdd, user_id=92b4a96c11dc49fd8ae2145fb6fef61e, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:13.625827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe81e06-4504-4828-b597-5af86ec85b9c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:13.640913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 955/1888] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:13 2022] PUT /identity/v3/projects/86f711618beb4eaa90b4b38bc938bbdd/users/92b4a96c11dc49fd8ae2145fb6fef61e/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:13.654691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fa6bd-3797-49d3-86d1-61cc1cb16875 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.655589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fa6bd-3797-49d3-86d1-61cc1cb16875 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.655826 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388fa6bd-3797-49d3-86d1-61cc1cb16875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.686631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-388fa6bd-3797-49d3-86d1-61cc1cb16875 None None] MFA Rules not processed for user `92b4a96c11dc49fd8ae2145fb6fef61e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:13.688008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-388fa6bd-3797-49d3-86d1-61cc1cb16875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:13.757878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 934/1889] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:13.763079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe81e06-4504-4828-b597-5af86ec85b9c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:13.829546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe81e06-4504-4828-b597-5af86ec85b9c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:13.834728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427a111-eaef-40c2-89ff-5ae141c78d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:13.835342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427a111-eaef-40c2-89ff-5ae141c78d5c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.835620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427a111-eaef-40c2-89ff-5ae141c78d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.835935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427a111-eaef-40c2-89ff-5ae141c78d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.836572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427a111-eaef-40c2-89ff-5ae141c78d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:13.838481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427a111-eaef-40c2-89ff-5ae141c78d5c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:13.871878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 956/1890] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:13 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:13.872541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fa6bd-3797-49d3-86d1-61cc1cb16875 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:13.884682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388fa6bd-3797-49d3-86d1-61cc1cb16875 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:13.930402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:13.945617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485ecc-e212-478a-9e31-f1c692aa4ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:13.946225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485ecc-e212-478a-9e31-f1c692aa4ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.946499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485ecc-e212-478a-9e31-f1c692aa4ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.946765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04485ecc-e212-478a-9e31-f1c692aa4ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.947418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04485ecc-e212-478a-9e31-f1c692aa4ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:13.949414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04485ecc-e212-478a-9e31-f1c692aa4ff9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:13.975130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 935/1891] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:13 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:13.976369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427a111-eaef-40c2-89ff-5ae141c78d5c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:13.987132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427a111-eaef-40c2-89ff-5ae141c78d5c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:13.991929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f4f03-a906-4924-80a5-a04a7ee49c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:13.992614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f4f03-a906-4924-80a5-a04a7ee49c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:13.992958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f4f03-a906-4924-80a5-a04a7ee49c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:13.993300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f4f03-a906-4924-80a5-a04a7ee49c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:13.994140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f4f03-a906-4924-80a5-a04a7ee49c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:13.995924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f4f03-a906-4924-80a5-a04a7ee49c02 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:14.011507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 957/1892] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:14.012865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485ecc-e212-478a-9e31-f1c692aa4ff9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:14.025437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04485ecc-e212-478a-9e31-f1c692aa4ff9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:14.031645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d60ddf-cf3c-4e41-b05c-b09e200d42f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:14.032480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d60ddf-cf3c-4e41-b05c-b09e200d42f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:14.032835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d60ddf-cf3c-4e41-b05c-b09e200d42f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:14.033178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d60ddf-cf3c-4e41-b05c-b09e200d42f3 admin admin] PATH_INFO: `/v3/projects/cc3aaffd47a444139171f772d1c43ea6/users/e39adf7ddc8148e38c82e98f5c5add04/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:14.057166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d60ddf-cf3c-4e41-b05c-b09e200d42f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc3aaffd47a444139171f772d1c43ea6, user_id=e39adf7ddc8148e38c82e98f5c5add04, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:14.060001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d60ddf-cf3c-4e41-b05c-b09e200d42f3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:14.083618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 936/1893] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:14 2022] PUT /identity/v3/projects/cc3aaffd47a444139171f772d1c43ea6/users/e39adf7ddc8148e38c82e98f5c5add04/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:14.087748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f4f03-a906-4924-80a5-a04a7ee49c02 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:14.098535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f4f03-a906-4924-80a5-a04a7ee49c02 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:14.157664 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632ea98-cb17-41a1-afc8-d59865b06c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:14.158277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632ea98-cb17-41a1-afc8-d59865b06c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:14.158559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632ea98-cb17-41a1-afc8-d59865b06c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:14.158834 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632ea98-cb17-41a1-afc8-d59865b06c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:14.159487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632ea98-cb17-41a1-afc8-d59865b06c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:14.161119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632ea98-cb17-41a1-afc8-d59865b06c34 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:14.172011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 958/1894] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:21:14.176253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d60ddf-cf3c-4e41-b05c-b09e200d42f3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:14.187588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d60ddf-cf3c-4e41-b05c-b09e200d42f3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:14.192230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa610992-3abf-4498-a4e7-e237501302b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:14.192897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa610992-3abf-4498-a4e7-e237501302b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:14.193175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa610992-3abf-4498-a4e7-e237501302b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:14.193450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa610992-3abf-4498-a4e7-e237501302b5 admin admin] PATH_INFO: `/v3/projects/cc3aaffd47a444139171f772d1c43ea6/users/e39adf7ddc8148e38c82e98f5c5add04/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:14.195348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa610992-3abf-4498-a4e7-e237501302b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc3aaffd47a444139171f772d1c43ea6, user_id=e39adf7ddc8148e38c82e98f5c5add04, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:14.198150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa610992-3abf-4498-a4e7-e237501302b5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:14.209871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 937/1895] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:14 2022] PUT /identity/v3/projects/cc3aaffd47a444139171f772d1c43ea6/users/e39adf7ddc8148e38c82e98f5c5add04/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:14.220709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcec81-b3c9-44b9-a875-b045aa70a1bf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:14.221083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcec81-b3c9-44b9-a875-b045aa70a1bf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:14.221425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcec81-b3c9-44b9-a875-b045aa70a1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:14.251395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-80dcec81-b3c9-44b9-a875-b045aa70a1bf None None] MFA Rules not processed for user `e39adf7ddc8148e38c82e98f5c5add04`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:14.252638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-80dcec81-b3c9-44b9-a875-b045aa70a1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:14.256685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:14.323175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 959/1896] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:14.335497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa610992-3abf-4498-a4e7-e237501302b5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:14.337213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-aa610992-3abf-4498-a4e7-e237501302b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:14.396189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa610992-3abf-4498-a4e7-e237501302b5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:14.399883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0e086-414f-4054-a957-931d96fb22d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:14.400493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0e086-414f-4054-a957-931d96fb22d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:14.400777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0e086-414f-4054-a957-931d96fb22d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:14.401048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0e086-414f-4054-a957-931d96fb22d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:14.411194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0e086-414f-4054-a957-931d96fb22d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:14.412715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0e086-414f-4054-a957-931d96fb22d1 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:14.423846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 938/1897] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:14.644895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcec81-b3c9-44b9-a875-b045aa70a1bf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:14.646304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-80dcec81-b3c9-44b9-a875-b045aa70a1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:14.658028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcec81-b3c9-44b9-a875-b045aa70a1bf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:14.661590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18259d12-6cd8-4552-ab03-f09b77cabbc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:14.662212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18259d12-6cd8-4552-ab03-f09b77cabbc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:14.662512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18259d12-6cd8-4552-ab03-f09b77cabbc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:14.662783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18259d12-6cd8-4552-ab03-f09b77cabbc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:14.720715 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18259d12-6cd8-4552-ab03-f09b77cabbc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:14.722150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18259d12-6cd8-4552-ab03-f09b77cabbc9 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:14.730916 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 960/1898] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:15.018817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0e086-414f-4054-a957-931d96fb22d1 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:15.081371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0e086-414f-4054-a957-931d96fb22d1 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:15.086252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:15.086823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:15.087108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:15.087382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] PATH_INFO: `/v3/users/92b4a96c11dc49fd8ae2145fb6fef61e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:15.088806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b4a96c11dc49fd8ae2145fb6fef61e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:15.090846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:15.142812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:15.144725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b4a96c11dc49fd8ae2145fb6fef61e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.145737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b4a96c11dc49fd8ae2145fb6fef61e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.154960 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b4a96c11dc49fd8ae2145fb6fef61e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.155348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b4a96c11dc49fd8ae2145fb6fef61e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.162820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b4a96c11dc49fd8ae2145fb6fef61e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.180672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 939/1899] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:15 2022] DELETE /identity/v3/users/92b4a96c11dc49fd8ae2145fb6fef61e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:15.185246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f1e81-ac91-487d-89ab-d10ba19b21f9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:15.185539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f1e81-ac91-487d-89ab-d10ba19b21f9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:15.185814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f1e81-ac91-487d-89ab-d10ba19b21f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:15.206070 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f39f1e81-ac91-487d-89ab-d10ba19b21f9 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:15.207282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f39f1e81-ac91-487d-89ab-d10ba19b21f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:15.268287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 961/1900] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:15.296054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:15.297665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:15.363459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd3110-af23-4e4f-a989-f36c6758e8ed admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:15.367720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16979a-3108-4f70-ad68-3f33aca42459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:15.368442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16979a-3108-4f70-ad68-3f33aca42459 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:15.368797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16979a-3108-4f70-ad68-3f33aca42459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:15.369143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16979a-3108-4f70-ad68-3f33aca42459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:15.379846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16979a-3108-4f70-ad68-3f33aca42459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:15.381287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16979a-3108-4f70-ad68-3f33aca42459 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:15.395497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 940/1901] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:15.681532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f1e81-ac91-487d-89ab-d10ba19b21f9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:15.683145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f39f1e81-ac91-487d-89ab-d10ba19b21f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:15.702724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f1e81-ac91-487d-89ab-d10ba19b21f9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:15.708453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:15.709475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:15.710017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:15.710441 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] PATH_INFO: `/v3/projects/86f711618beb4eaa90b4b38bc938bbdd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:15.711935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f711618beb4eaa90b4b38bc938bbdd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:15.714387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:15.779545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:15.794692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f711618beb4eaa90b4b38bc938bbdd', 'request_id': 'req-3618a9b9-bf1c-434c-9747-193ff9544ecb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.802807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f711618beb4eaa90b4b38bc938bbdd', 'request_id': 'req-3618a9b9-bf1c-434c-9747-193ff9544ecb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.818781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f711618beb4eaa90b4b38bc938bbdd', 'request_id': 'req-3618a9b9-bf1c-434c-9747-193ff9544ecb'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.833202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] The token cache is being invalidate because project 86f711618beb4eaa90b4b38bc938bbdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:15.833491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.834163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.835700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 962/1902] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:15 2022] DELETE /identity/v3/projects/86f711618beb4eaa90b4b38bc938bbdd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:15.839729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16979a-3108-4f70-ad68-3f33aca42459 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:15.849935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16979a-3108-4f70-ad68-3f33aca42459 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:15.913877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:15.914562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:15.914850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:15.915169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] PATH_INFO: `/v3/users/e39adf7ddc8148e38c82e98f5c5add04` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:15.916338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39adf7ddc8148e38c82e98f5c5add04)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:15.918292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:15.992024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39adf7ddc8148e38c82e98f5c5add04'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:15.992789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39adf7ddc8148e38c82e98f5c5add04'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:16.002311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39adf7ddc8148e38c82e98f5c5add04'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:16.002953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39adf7ddc8148e38c82e98f5c5add04'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:16.011165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d7fea244-9032-409e-a28a-e34a7d7c071e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39adf7ddc8148e38c82e98f5c5add04'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:16.013603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:16.051877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 941/1903] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:15 2022] DELETE /identity/v3/users/e39adf7ddc8148e38c82e98f5c5add04 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:16.444202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:16.446163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:16.502272 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618a9b9-bf1c-434c-9747-193ff9544ecb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:16.613257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:16.614506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:16.614506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:16.614843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] PATH_INFO: `/v3/projects/cc3aaffd47a444139171f772d1c43ea6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:16.616437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc3aaffd47a444139171f772d1c43ea6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:16.618732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:16.685938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc3aaffd47a444139171f772d1c43ea6', 'request_id': 'req-84a5fb82-c719-4f74-a27d-bac871f7d24f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:16.693237 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc3aaffd47a444139171f772d1c43ea6', 'request_id': 'req-84a5fb82-c719-4f74-a27d-bac871f7d24f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:16.703861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc3aaffd47a444139171f772d1c43ea6', 'request_id': 'req-84a5fb82-c719-4f74-a27d-bac871f7d24f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:16.716656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] The token cache is being invalidate because project cc3aaffd47a444139171f772d1c43ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:16.716945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:16.718287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-84a5fb82-c719-4f74-a27d-bac871f7d24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:16.719719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 963/1904] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:16 2022] DELETE /identity/v3/projects/cc3aaffd47a444139171f772d1c43ea6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:16.734078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b34bec-ee29-4e01-a1cd-d96d7ba12475 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:16.734669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b34bec-ee29-4e01-a1cd-d96d7ba12475 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:16.734669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b34bec-ee29-4e01-a1cd-d96d7ba12475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:16.755497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d7b34bec-ee29-4e01-a1cd-d96d7ba12475 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:16.756864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d7b34bec-ee29-4e01-a1cd-d96d7ba12475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:16.844564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 942/1905] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:16.852679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e3c32-bb57-4b05-ad2e-bf45585fae93 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:16.852967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e3c32-bb57-4b05-ad2e-bf45585fae93 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:16.853240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e3c32-bb57-4b05-ad2e-bf45585fae93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:16.875103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-008e3c32-bb57-4b05-ad2e-bf45585fae93 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:16.876711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-008e3c32-bb57-4b05-ad2e-bf45585fae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:16.885793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 964/1906] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:16.890589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b34bec-ee29-4e01-a1cd-d96d7ba12475 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:16.916429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b34bec-ee29-4e01-a1cd-d96d7ba12475 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:16.925687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dc704-e8e0-46bd-9512-8518f88dba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:16.926567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dc704-e8e0-46bd-9512-8518f88dba5f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:16.926855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dc704-e8e0-46bd-9512-8518f88dba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:16.927165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42dc704-e8e0-46bd-9512-8518f88dba5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:16.928030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dc704-e8e0-46bd-9512-8518f88dba5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:21:16.928350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dc704-e8e0-46bd-9512-8518f88dba5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:16.930408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42dc704-e8e0-46bd-9512-8518f88dba5f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:16.953938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 943/1907] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:21:16.958687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e3c32-bb57-4b05-ad2e-bf45585fae93 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:16.968632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e3c32-bb57-4b05-ad2e-bf45585fae93 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:16.973513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225d52a-7a80-4892-88f4-abfc430185de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:16.974119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225d52a-7a80-4892-88f4-abfc430185de admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:16.974404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225d52a-7a80-4892-88f4-abfc430185de admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:16.974676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225d52a-7a80-4892-88f4-abfc430185de admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:16.975302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225d52a-7a80-4892-88f4-abfc430185de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:16.977973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225d52a-7a80-4892-88f4-abfc430185de admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:17.029175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 965/1908] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:16 2022] POST /identity/v3/projects => generated 361 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:17.034833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dc704-e8e0-46bd-9512-8518f88dba5f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:17.046520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42dc704-e8e0-46bd-9512-8518f88dba5f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:17.136699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225d52a-7a80-4892-88f4-abfc430185de admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:17.138515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9225d52a-7a80-4892-88f4-abfc430185de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:17.155713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd42b1-a9ba-4c0e-bbed-054ca1adf6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:17.156461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd42b1-a9ba-4c0e-bbed-054ca1adf6d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:17.156816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd42b1-a9ba-4c0e-bbed-054ca1adf6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:17.157179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd42b1-a9ba-4c0e-bbed-054ca1adf6d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:17.157927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd42b1-a9ba-4c0e-bbed-054ca1adf6d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:17.160009 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd42b1-a9ba-4c0e-bbed-054ca1adf6d0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:17.189573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 944/1909] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:17 2022] POST /identity/v3/users => generated 281 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:17.193915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd42b1-a9ba-4c0e-bbed-054ca1adf6d0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:17.208693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd42b1-a9ba-4c0e-bbed-054ca1adf6d0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:17.215056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdd304-cda3-4b9b-974a-0079fbece52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:17.216014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdd304-cda3-4b9b-974a-0079fbece52a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:17.216502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdd304-cda3-4b9b-974a-0079fbece52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:17.216980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfdd304-cda3-4b9b-974a-0079fbece52a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:17.217737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdd304-cda3-4b9b-974a-0079fbece52a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:17.218624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:17.219710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfdd304-cda3-4b9b-974a-0079fbece52a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:17.237235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 945/1910] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:21:17.249280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdd304-cda3-4b9b-974a-0079fbece52a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:17.266189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfdd304-cda3-4b9b-974a-0079fbece52a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:17.278358 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8624c-3512-4ae3-bf80-f49c49f75747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:17.279085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8624c-3512-4ae3-bf80-f49c49f75747 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:17.279441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8624c-3512-4ae3-bf80-f49c49f75747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:17.279939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8624c-3512-4ae3-bf80-f49c49f75747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:17.280609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8624c-3512-4ae3-bf80-f49c49f75747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:17.282190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8624c-3512-4ae3-bf80-f49c49f75747 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:17.282572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225d52a-7a80-4892-88f4-abfc430185de admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:17.287630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc1b64-87a7-421d-ad38-74e4b4c61173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:17.288768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc1b64-87a7-421d-ad38-74e4b4c61173 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:17.289058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc1b64-87a7-421d-ad38-74e4b4c61173 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:17.289345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc1b64-87a7-421d-ad38-74e4b4c61173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:17.292134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a5bc1b64-87a7-421d-ad38-74e4b4c61173 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:17.293309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 946/1911] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:17.297784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8624c-3512-4ae3-bf80-f49c49f75747 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:17.308249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8624c-3512-4ae3-bf80-f49c49f75747 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:17.315276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07f778-9c90-42e9-8070-4cf30482cf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:17.316461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07f778-9c90-42e9-8070-4cf30482cf03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:17.316461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07f778-9c90-42e9-8070-4cf30482cf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:17.316669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07f778-9c90-42e9-8070-4cf30482cf03 admin admin] PATH_INFO: `/v3/projects/28774148c6774bc3abff731ffad168d3/users/cda1b56debc543839ecbc23a8f0632f7/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:17.337044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07f778-9c90-42e9-8070-4cf30482cf03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28774148c6774bc3abff731ffad168d3, user_id=cda1b56debc543839ecbc23a8f0632f7, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:17.339706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07f778-9c90-42e9-8070-4cf30482cf03 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:17.367800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 947/1912] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:17 2022] PUT /identity/v3/projects/28774148c6774bc3abff731ffad168d3/users/cda1b56debc543839ecbc23a8f0632f7/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:21:17.374483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d992f9-4b72-41ac-b546-44a155e53165 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:17.374794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d992f9-4b72-41ac-b546-44a155e53165 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:17.375121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d992f9-4b72-41ac-b546-44a155e53165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:17.398810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc1b64-87a7-421d-ad38-74e4b4c61173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:17.400913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc1b64-87a7-421d-ad38-74e4b4c61173 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:17.414930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 966/1913] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:21:17.416111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:17.420400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-49d992f9-4b72-41ac-b546-44a155e53165 None None] MFA Rules not processed for user `cda1b56debc543839ecbc23a8f0632f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:17.422527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-49d992f9-4b72-41ac-b546-44a155e53165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:17.525609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 948/1914] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:17.609620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc1b64-87a7-421d-ad38-74e4b4c61173 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:17.621774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc1b64-87a7-421d-ad38-74e4b4c61173 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:17.712903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd828a8b-fdfd-4fac-8936-f2fd8d375ca6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:17.713620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd828a8b-fdfd-4fac-8936-f2fd8d375ca6 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:17.713796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd828a8b-fdfd-4fac-8936-f2fd8d375ca6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:17.714064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd828a8b-fdfd-4fac-8936-f2fd8d375ca6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:17.738993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd828a8b-fdfd-4fac-8936-f2fd8d375ca6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:17.740611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd828a8b-fdfd-4fac-8936-f2fd8d375ca6 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:17.770570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 967/1915] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:17.798053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d992f9-4b72-41ac-b546-44a155e53165 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:17.929791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d992f9-4b72-41ac-b546-44a155e53165 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:17.944664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:17.946139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:17.946573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:17.946821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] PATH_INFO: `/v3/users/cda1b56debc543839ecbc23a8f0632f7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:17.948896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda1b56debc543839ecbc23a8f0632f7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:17.969988 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:18.122489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda1b56debc543839ecbc23a8f0632f7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.126503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda1b56debc543839ecbc23a8f0632f7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.149944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda1b56debc543839ecbc23a8f0632f7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.150695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda1b56debc543839ecbc23a8f0632f7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.163190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda1b56debc543839ecbc23a8f0632f7'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.220065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 949/1916] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:17 2022] DELETE /identity/v3/users/cda1b56debc543839ecbc23a8f0632f7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:18.221510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87078240-4d4c-4c8f-a7ec-cfff8e3d8b1e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:18.221796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87078240-4d4c-4c8f-a7ec-cfff8e3d8b1e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:18.222070 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87078240-4d4c-4c8f-a7ec-cfff8e3d8b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:18.247314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-87078240-4d4c-4c8f-a7ec-cfff8e3d8b1e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:18.253851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-87078240-4d4c-4c8f-a7ec-cfff8e3d8b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:18.377798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 968/1917] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:18.389236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:18.391227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:18.498355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:18.529276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87078240-4d4c-4c8f-a7ec-cfff8e3d8b1e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:18.535708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-87078240-4d4c-4c8f-a7ec-cfff8e3d8b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:18.559746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87078240-4d4c-4c8f-a7ec-cfff8e3d8b1e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:18.564034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8075926-f24b-479a-82fc-7a28288ac62b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:18.567671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:18.568656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:18.569253 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:18.570047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] PATH_INFO: `/v3/users/bd7b747c782e41f893f9c45db1494e1b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:18.572010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7b747c782e41f893f9c45db1494e1b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:18.575010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:18.575224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882110b-6d68-464d-b82a-034f84ff4e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:18.575885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882110b-6d68-464d-b82a-034f84ff4e52 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:18.576171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882110b-6d68-464d-b82a-034f84ff4e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:18.576446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882110b-6d68-464d-b82a-034f84ff4e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:18.600170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:18.601198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882110b-6d68-464d-b82a-034f84ff4e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:18.602711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882110b-6d68-464d-b82a-034f84ff4e52 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:18.619745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 950/1918] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:21:18.669469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7b747c782e41f893f9c45db1494e1b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.680007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7b747c782e41f893f9c45db1494e1b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.681186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7b747c782e41f893f9c45db1494e1b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.682110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7b747c782e41f893f9c45db1494e1b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.691652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7b747c782e41f893f9c45db1494e1b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:18.736966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 969/1919] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:18 2022] DELETE /identity/v3/users/bd7b747c782e41f893f9c45db1494e1b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:18.741611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedaa63-b7f0-4853-a6dd-acabf12d6da8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:18.741895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedaa63-b7f0-4853-a6dd-acabf12d6da8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:18.742277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedaa63-b7f0-4853-a6dd-acabf12d6da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:18.769906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3cedaa63-b7f0-4853-a6dd-acabf12d6da8 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:18.771601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3cedaa63-b7f0-4853-a6dd-acabf12d6da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:18.909775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 951/1920] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:18.911555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:18.913004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:18.990690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedaa63-b7f0-4853-a6dd-acabf12d6da8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:18.991529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3cedaa63-b7f0-4853-a6dd-acabf12d6da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:19.017370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cedaa63-b7f0-4853-a6dd-acabf12d6da8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:19.029522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:19.033344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:19.033749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:19.033970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] PATH_INFO: `/v3/users/03376fdb8a3b4cda9aaee54294fe3766` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:19.035955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=03376fdb8a3b4cda9aaee54294fe3766)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:19.038037 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:19.051656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d61c-051f-4b7a-af1a-4f2eea8d3fad admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:19.063314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525a18-5d77-4ebc-8c3e-399603affd56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:19.064659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525a18-5d77-4ebc-8c3e-399603affd56 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:19.065248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525a18-5d77-4ebc-8c3e-399603affd56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:19.065700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525a18-5d77-4ebc-8c3e-399603affd56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:19.095859 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525a18-5d77-4ebc-8c3e-399603affd56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:19.099689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525a18-5d77-4ebc-8c3e-399603affd56 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:19.115208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 970/1921] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:19.126826 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03376fdb8a3b4cda9aaee54294fe3766'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.127604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03376fdb8a3b4cda9aaee54294fe3766'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.148206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03376fdb8a3b4cda9aaee54294fe3766'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.154465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03376fdb8a3b4cda9aaee54294fe3766'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.181550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-36dd7055-4756-49d1-97fc-5971915f33ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03376fdb8a3b4cda9aaee54294fe3766'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.243577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525a18-5d77-4ebc-8c3e-399603affd56 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:19.245603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-37525a18-5d77-4ebc-8c3e-399603affd56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:19.260331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 952/1922] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:18 2022] DELETE /identity/v3/users/03376fdb8a3b4cda9aaee54294fe3766 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:19.267703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61a963-6c51-48b2-bfab-a37d97dea6df None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:19.267920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61a963-6c51-48b2-bfab-a37d97dea6df None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:19.268190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61a963-6c51-48b2-bfab-a37d97dea6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:19.286428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525a18-5d77-4ebc-8c3e-399603affd56 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:19.312810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fb61a963-6c51-48b2-bfab-a37d97dea6df None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:19.315088 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fb61a963-6c51-48b2-bfab-a37d97dea6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:19.364334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:19.408503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 953/1923] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:19.424348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61a963-6c51-48b2-bfab-a37d97dea6df None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:19.427535 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fb61a963-6c51-48b2-bfab-a37d97dea6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:19.463016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:19.466042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:19.468059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:19.469146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] PATH_INFO: `/v3/projects/28774148c6774bc3abff731ffad168d3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:19.472363 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28774148c6774bc3abff731ffad168d3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:19.476760 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:19.546394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:19.568698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61a963-6c51-48b2-bfab-a37d97dea6df None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:19.581723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28774148c6774bc3abff731ffad168d3', 'request_id': 'req-61710f78-3bc6-425f-943e-bb306d5bc799'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.592190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28774148c6774bc3abff731ffad168d3', 'request_id': 'req-61710f78-3bc6-425f-943e-bb306d5bc799'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.604440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28774148c6774bc3abff731ffad168d3', 'request_id': 'req-61710f78-3bc6-425f-943e-bb306d5bc799'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.619444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] The token cache is being invalidate because project 28774148c6774bc3abff731ffad168d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:19.619924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.620713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-61710f78-3bc6-425f-943e-bb306d5bc799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:19.622689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 971/1924] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:19 2022] DELETE /identity/v3/projects/28774148c6774bc3abff731ffad168d3 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:21:19.628003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4de033-4036-46d6-a5dc-e9828bd75c79 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:19.628981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4de033-4036-46d6-a5dc-e9828bd75c79 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:19.629406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4de033-4036-46d6-a5dc-e9828bd75c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:19.660451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a2e67-8887-415b-b322-22cc314c70e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:19.661694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a2e67-8887-415b-b322-22cc314c70e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:19.662267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a2e67-8887-415b-b322-22cc314c70e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:19.662703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a2e67-8887-415b-b322-22cc314c70e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:19.664698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3e4de033-4036-46d6-a5dc-e9828bd75c79 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:19.665119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9f0a2e67-8887-415b-b322-22cc314c70e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:19.667100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3e4de033-4036-46d6-a5dc-e9828bd75c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:19.794717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a2e67-8887-415b-b322-22cc314c70e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:19.796093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 972/1925] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:19.797004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a2e67-8887-415b-b322-22cc314c70e8 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:19.803461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08666458-8541-4934-b24f-00707b67b1c4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:19.804073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08666458-8541-4934-b24f-00707b67b1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:19.804422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08666458-8541-4934-b24f-00707b67b1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:19.811577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 954/1926] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:19.830594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-08666458-8541-4934-b24f-00707b67b1c4 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:19.832401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-08666458-8541-4934-b24f-00707b67b1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:19.842009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 973/1927] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:19.846654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a2e67-8887-415b-b322-22cc314c70e8 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:19.858563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a2e67-8887-415b-b322-22cc314c70e8 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:19.864175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b9c19-6ad1-4b30-88d5-4d6ea2e30967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:19.864811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b9c19-6ad1-4b30-88d5-4d6ea2e30967 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:19.865096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b9c19-6ad1-4b30-88d5-4d6ea2e30967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:19.865539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b9c19-6ad1-4b30-88d5-4d6ea2e30967 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:19.866204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b9c19-6ad1-4b30-88d5-4d6ea2e30967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:21:19.866509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b9c19-6ad1-4b30-88d5-4d6ea2e30967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:19.868343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b9c19-6ad1-4b30-88d5-4d6ea2e30967 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:19.888276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 955/1928] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:21:19.890410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08666458-8541-4934-b24f-00707b67b1c4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:19.904266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08666458-8541-4934-b24f-00707b67b1c4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:19.911230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ded75-bb4c-4ec1-83bf-b069dae13a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:19.912621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ded75-bb4c-4ec1-83bf-b069dae13a81 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:19.912747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ded75-bb4c-4ec1-83bf-b069dae13a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:19.913042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ded75-bb4c-4ec1-83bf-b069dae13a81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:19.913757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ded75-bb4c-4ec1-83bf-b069dae13a81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:19.915854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ded75-bb4c-4ec1-83bf-b069dae13a81 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:19.950666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 974/1929] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:19 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:21:19.954796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b9c19-6ad1-4b30-88d5-4d6ea2e30967 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:19.965964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b9c19-6ad1-4b30-88d5-4d6ea2e30967 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.009130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:20.044555 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ddb68-f8a1-4741-beef-021b0b26f5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:20.045089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ddb68-f8a1-4741-beef-021b0b26f5b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.046021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ddb68-f8a1-4741-beef-021b0b26f5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.046297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ddb68-f8a1-4741-beef-021b0b26f5b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.046940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ddb68-f8a1-4741-beef-021b0b26f5b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:20.049125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ddb68-f8a1-4741-beef-021b0b26f5b4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:20.052549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ded75-bb4c-4ec1-83bf-b069dae13a81 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:20.054781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d62ded75-bb4c-4ec1-83bf-b069dae13a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:20.073344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ded75-bb4c-4ec1-83bf-b069dae13a81 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.075988 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 956/1930] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:19 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:21:20.077457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ddb68-f8a1-4741-beef-021b0b26f5b4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:20.079474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:20.080186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.080465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.080739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] PATH_INFO: `/v3/projects/a337d07daf7740b1b9fb7062a7510153` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.082027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a337d07daf7740b1b9fb7062a7510153)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:20.084517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:20.099087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ddb68-f8a1-4741-beef-021b0b26f5b4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.115204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3144a6-bb73-4731-ab97-4ceed5ab67de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:20.116281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3144a6-bb73-4731-ab97-4ceed5ab67de admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.117669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3144a6-bb73-4731-ab97-4ceed5ab67de admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.118766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3144a6-bb73-4731-ab97-4ceed5ab67de admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.119570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3144a6-bb73-4731-ab97-4ceed5ab67de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:20.124428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3144a6-bb73-4731-ab97-4ceed5ab67de admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:20.138986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 957/1931] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:21:20.142954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3144a6-bb73-4731-ab97-4ceed5ab67de admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:20.154062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a337d07daf7740b1b9fb7062a7510153', 'request_id': 'req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.154247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3144a6-bb73-4731-ab97-4ceed5ab67de admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.161890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a337d07daf7740b1b9fb7062a7510153', 'request_id': 'req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.173721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a337d07daf7740b1b9fb7062a7510153', 'request_id': 'req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.191650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] The token cache is being invalidate because project a337d07daf7740b1b9fb7062a7510153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:20.191984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.192725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.194889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 975/1932] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:20 2022] DELETE /identity/v3/projects/a337d07daf7740b1b9fb7062a7510153 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:20.195263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:20.196676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:20.224503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9791cf3-4f47-43c9-92cd-977813999760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:20.225153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9791cf3-4f47-43c9-92cd-977813999760 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.225435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9791cf3-4f47-43c9-92cd-977813999760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.225708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9791cf3-4f47-43c9-92cd-977813999760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.226357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9791cf3-4f47-43c9-92cd-977813999760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:20.226620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:20.228005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9791cf3-4f47-43c9-92cd-977813999760 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:20.242406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 958/1933] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:21:20.242406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9791cf3-4f47-43c9-92cd-977813999760 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:20.242634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e9791cf3-4f47-43c9-92cd-977813999760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:20.263123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9791cf3-4f47-43c9-92cd-977813999760 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.268200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:20.268839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.269123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.269418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] PATH_INFO: `/v3/users/7b8afab10f8e4f85b73a6a5a5b763136` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.270546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8afab10f8e4f85b73a6a5a5b763136)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:20.272515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:20.284295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b7862-43e1-4d32-b7f7-89ea00e4dfb5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.289601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:20.290239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.290513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.290780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] PATH_INFO: `/v3/projects/335f7a2759b74da186531f8293357695` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.291928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=335f7a2759b74da186531f8293357695)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:20.293828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:20.327972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8afab10f8e4f85b73a6a5a5b763136'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.328698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8afab10f8e4f85b73a6a5a5b763136'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.336867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8afab10f8e4f85b73a6a5a5b763136'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.337669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8afab10f8e4f85b73a6a5a5b763136'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.344549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335f7a2759b74da186531f8293357695', 'request_id': 'req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.345590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8afab10f8e4f85b73a6a5a5b763136'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.350709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335f7a2759b74da186531f8293357695', 'request_id': 'req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.359047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335f7a2759b74da186531f8293357695', 'request_id': 'req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.364539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 959/1934] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:20 2022] DELETE /identity/v3/users/7b8afab10f8e4f85b73a6a5a5b763136 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:20.365265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:20.368797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:20.370603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] The token cache is being invalidate because project 335f7a2759b74da186531f8293357695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:20.370899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.371802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.374028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 976/1935] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:20 2022] DELETE /identity/v3/projects/335f7a2759b74da186531f8293357695 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:20.377702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:20.379062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:20.396980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:20.454427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbd4f0-3050-4460-af7d-df7013a85c35 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.459665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cc9a7-2302-44d1-9bf7-44a8ed81dd74 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.462933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27476ce-faf5-4be6-a713-898c542422e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:20.463652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27476ce-faf5-4be6-a713-898c542422e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.463984 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27476ce-faf5-4be6-a713-898c542422e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.464253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27476ce-faf5-4be6-a713-898c542422e0 admin admin] PATH_INFO: `/v3/projects/c61099572cb04210988b5863247bc641/users/dd5fd430044c4fd39f08a5086bcb69ba/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.468936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:20.469702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.469983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.470328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] PATH_INFO: `/v3/users/7d4fc26327714fe2abce6fa16b837b11` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.471415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4fc26327714fe2abce6fa16b837b11)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:20.473383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:20.481566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27476ce-faf5-4be6-a713-898c542422e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61099572cb04210988b5863247bc641, user_id=dd5fd430044c4fd39f08a5086bcb69ba, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:20.485045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27476ce-faf5-4be6-a713-898c542422e0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:20.502163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 960/1936] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:20 2022] PUT /identity/v3/projects/c61099572cb04210988b5863247bc641/users/dd5fd430044c4fd39f08a5086bcb69ba/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:20.514939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9b8fc-8003-4805-a93a-e38585a27c00 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.515408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9b8fc-8003-4805-a93a-e38585a27c00 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.515595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9b8fc-8003-4805-a93a-e38585a27c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.544574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4fc26327714fe2abce6fa16b837b11'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.548881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:20.550224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-81d9b8fc-8003-4805-a93a-e38585a27c00 None None] MFA Rules not processed for user `dd5fd430044c4fd39f08a5086bcb69ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:20.551688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-81d9b8fc-8003-4805-a93a-e38585a27c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:20.557299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4fc26327714fe2abce6fa16b837b11'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.558269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4fc26327714fe2abce6fa16b837b11'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.559035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4fc26327714fe2abce6fa16b837b11'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.569998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4fc26327714fe2abce6fa16b837b11'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:20.599066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 977/1937] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:20 2022] DELETE /identity/v3/users/7d4fc26327714fe2abce6fa16b837b11 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:21:20.652226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 961/1938] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:20.665235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:20.666660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:20.742083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034bee5-765d-4fae-bc03-0f94e1daee4d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.756359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0cc89-2f5e-4066-b34d-a5138a4367b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:20.756850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0cc89-2f5e-4066-b34d-a5138a4367b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:20.757134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0cc89-2f5e-4066-b34d-a5138a4367b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:20.757414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0cc89-2f5e-4066-b34d-a5138a4367b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:20.765228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9b8fc-8003-4805-a93a-e38585a27c00 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:20.766877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-81d9b8fc-8003-4805-a93a-e38585a27c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:20.855434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0cc89-2f5e-4066-b34d-a5138a4367b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:20.857342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0cc89-2f5e-4066-b34d-a5138a4367b3 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:20.885889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 978/1939] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:20.906941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9b8fc-8003-4805-a93a-e38585a27c00 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:20.999361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0cc89-2f5e-4066-b34d-a5138a4367b3 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.001244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-22f0cc89-2f5e-4066-b34d-a5138a4367b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:21.006941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:21.080456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.081356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.081736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.082166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] PATH_INFO: `/v3/projects/74d1c3287f3b497d8d5f8a3969db642e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.088716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d1c3287f3b497d8d5f8a3969db642e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.090871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.145922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0cc89-2f5e-4066-b34d-a5138a4367b3 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.171376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d1c3287f3b497d8d5f8a3969db642e', 'request_id': 'req-728306a1-0f0a-4d4c-9566-bde21a406e98'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.182058 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d1c3287f3b497d8d5f8a3969db642e', 'request_id': 'req-728306a1-0f0a-4d4c-9566-bde21a406e98'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.195345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d1c3287f3b497d8d5f8a3969db642e', 'request_id': 'req-728306a1-0f0a-4d4c-9566-bde21a406e98'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.213146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] The token cache is being invalidate because project 74d1c3287f3b497d8d5f8a3969db642e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:21.213146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.213146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-728306a1-0f0a-4d4c-9566-bde21a406e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.214495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 962/1940] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:20 2022] DELETE /identity/v3/projects/74d1c3287f3b497d8d5f8a3969db642e => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:21:21.215794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:21.219692 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23165640-5c10-448d-87ca-5ed21b89c3ee None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.220301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23165640-5c10-448d-87ca-5ed21b89c3ee None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.222430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23165640-5c10-448d-87ca-5ed21b89c3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.247423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-23165640-5c10-448d-87ca-5ed21b89c3ee None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:21.250278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-23165640-5c10-448d-87ca-5ed21b89c3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:21.277112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.278051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.278499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.278949 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] PATH_INFO: `/v3/projects/73ed7794d48c4878957afe8b2bb168fa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.281189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ed7794d48c4878957afe8b2bb168fa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.283560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.374253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 963/1941] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:21.374690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ed7794d48c4878957afe8b2bb168fa', 'request_id': 'req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.394136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ed7794d48c4878957afe8b2bb168fa', 'request_id': 'req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.397569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab079265-c78a-48e4-a893-aabd71ded1fd None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.397569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab079265-c78a-48e4-a893-aabd71ded1fd None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.398459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab079265-c78a-48e4-a893-aabd71ded1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.414053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ed7794d48c4878957afe8b2bb168fa', 'request_id': 'req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.427118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] The token cache is being invalidate because project 73ed7794d48c4878957afe8b2bb168fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:21.427561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.428693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bdd4e024-b9a8-4165-a0b6-b55e6e9afac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:21.430877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 979/1942] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:20 2022] DELETE /identity/v3/projects/73ed7794d48c4878957afe8b2bb168fa => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:21.435397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee834a1-633e-4bcc-871a-5b760b83e462 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.435858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee834a1-633e-4bcc-871a-5b760b83e462 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.436371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee834a1-633e-4bcc-871a-5b760b83e462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.480208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ab079265-c78a-48e4-a893-aabd71ded1fd None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:21.480805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2ee834a1-633e-4bcc-871a-5b760b83e462 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:21.481506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ab079265-c78a-48e4-a893-aabd71ded1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:21.482160 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2ee834a1-633e-4bcc-871a-5b760b83e462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:21.559774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 980/1943] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:21.564197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 964/1944] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:21.568389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5d57f-2afc-4cac-99b7-17565b7a2f25 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.568751 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5d57f-2afc-4cac-99b7-17565b7a2f25 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.568847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab079265-c78a-48e4-a893-aabd71ded1fd None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.569177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5d57f-2afc-4cac-99b7-17565b7a2f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.579040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab079265-c78a-48e4-a893-aabd71ded1fd None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.589562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8dcb5-63db-4b5c-8689-55670ded74a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.590205 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8dcb5-63db-4b5c-8689-55670ded74a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.590483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8dcb5-63db-4b5c-8689-55670ded74a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.590778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8dcb5-63db-4b5c-8689-55670ded74a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.591424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8dcb5-63db-4b5c-8689-55670ded74a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:21:21.591732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8dcb5-63db-4b5c-8689-55670ded74a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.593094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-afb5d57f-2afc-4cac-99b7-17565b7a2f25 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:21.594549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-afb5d57f-2afc-4cac-99b7-17565b7a2f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:21.597304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8dcb5-63db-4b5c-8689-55670ded74a1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.601835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:21.612610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 981/1945] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:21.616529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5d57f-2afc-4cac-99b7-17565b7a2f25 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.621633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 965/1946] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:21:21.622592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8dcb5-63db-4b5c-8689-55670ded74a1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.634638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5d57f-2afc-4cac-99b7-17565b7a2f25 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.637379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8dcb5-63db-4b5c-8689-55670ded74a1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.640647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841a972-a67c-45ea-9285-f0f2d3ea1e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.641564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841a972-a67c-45ea-9285-f0f2d3ea1e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.641838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841a972-a67c-45ea-9285-f0f2d3ea1e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.642109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841a972-a67c-45ea-9285-f0f2d3ea1e44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.642767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841a972-a67c-45ea-9285-f0f2d3ea1e44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:21:21.643071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841a972-a67c-45ea-9285-f0f2d3ea1e44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.645209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841a972-a67c-45ea-9285-f0f2d3ea1e44 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.651374 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0001b3-486e-4740-a52f-9fde2255921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.652151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0001b3-486e-4740-a52f-9fde2255921d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.652671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0001b3-486e-4740-a52f-9fde2255921d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.653017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0001b3-486e-4740-a52f-9fde2255921d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.653727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0001b3-486e-4740-a52f-9fde2255921d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.655871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0001b3-486e-4740-a52f-9fde2255921d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.659036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 982/1947] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:21:21.663880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841a972-a67c-45ea-9285-f0f2d3ea1e44 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.674580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841a972-a67c-45ea-9285-f0f2d3ea1e44 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.680118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7fab2-de01-43f6-ab45-af60e74d7396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.680833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7fab2-de01-43f6-ab45-af60e74d7396 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.681119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7fab2-de01-43f6-ab45-af60e74d7396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.681394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab7fab2-de01-43f6-ab45-af60e74d7396 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.682064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7fab2-de01-43f6-ab45-af60e74d7396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.684021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab7fab2-de01-43f6-ab45-af60e74d7396 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.687308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 966/1948] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:21 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:21.691670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0001b3-486e-4740-a52f-9fde2255921d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.718135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 983/1949] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:21 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:21:21.724823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7fab2-de01-43f6-ab45-af60e74d7396 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.749345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:21.778844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0001b3-486e-4740-a52f-9fde2255921d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.800877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab7fab2-de01-43f6-ab45-af60e74d7396 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.806299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b935f5-e938-43e6-822a-df7f9dee167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.806932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b935f5-e938-43e6-822a-df7f9dee167c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.807208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b935f5-e938-43e6-822a-df7f9dee167c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.807477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b935f5-e938-43e6-822a-df7f9dee167c admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.808151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b935f5-e938-43e6-822a-df7f9dee167c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.810254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b935f5-e938-43e6-822a-df7f9dee167c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.841872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00145e-38d1-441a-863a-00376a89d01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.846240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00145e-38d1-441a-863a-00376a89d01e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.846240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00145e-38d1-441a-863a-00376a89d01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.846240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00145e-38d1-441a-863a-00376a89d01e admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.846240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00145e-38d1-441a-863a-00376a89d01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.846588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00145e-38d1-441a-863a-00376a89d01e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.847096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 984/1950] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:21 2022] POST /identity/v3/users => generated 283 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:21.848725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b935f5-e938-43e6-822a-df7f9dee167c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.860198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b935f5-e938-43e6-822a-df7f9dee167c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.866478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e701d5-7bb1-436a-9d0f-264287f6060f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.867448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e701d5-7bb1-436a-9d0f-264287f6060f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.868006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e701d5-7bb1-436a-9d0f-264287f6060f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.868752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e701d5-7bb1-436a-9d0f-264287f6060f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.869425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e701d5-7bb1-436a-9d0f-264287f6060f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.871090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e701d5-7bb1-436a-9d0f-264287f6060f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.883817 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 985/1951] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:21.887587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e701d5-7bb1-436a-9d0f-264287f6060f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.894867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 967/1952] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:21 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:21:21.896566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00145e-38d1-441a-863a-00376a89d01e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.907170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e701d5-7bb1-436a-9d0f-264287f6060f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.912596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a572dde-34b9-4d5a-b481-be251ec1687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.913295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a572dde-34b9-4d5a-b481-be251ec1687c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.913644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a572dde-34b9-4d5a-b481-be251ec1687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.914021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a572dde-34b9-4d5a-b481-be251ec1687c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.914604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00145e-38d1-441a-863a-00376a89d01e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.914877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a572dde-34b9-4d5a-b481-be251ec1687c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.916637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a572dde-34b9-4d5a-b481-be251ec1687c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.920161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f23c27-9989-4f24-962d-5a46f5d1935e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.920804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f23c27-9989-4f24-962d-5a46f5d1935e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.921087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f23c27-9989-4f24-962d-5a46f5d1935e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.921361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f23c27-9989-4f24-962d-5a46f5d1935e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.922020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f23c27-9989-4f24-962d-5a46f5d1935e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.923851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f23c27-9989-4f24-962d-5a46f5d1935e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.929103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 986/1953] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:21.943077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 968/1954] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:21.944324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f23c27-9989-4f24-962d-5a46f5d1935e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.944459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a572dde-34b9-4d5a-b481-be251ec1687c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:21.956682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f23c27-9989-4f24-962d-5a46f5d1935e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.966438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a572dde-34b9-4d5a-b481-be251ec1687c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:21.969958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc9bab-e130-4940-837e-e5cd3c032612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.970596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc9bab-e130-4940-837e-e5cd3c032612 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.970957 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc9bab-e130-4940-837e-e5cd3c032612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.971234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc9bab-e130-4940-837e-e5cd3c032612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.972052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc9bab-e130-4940-837e-e5cd3c032612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:21.972601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dc1d4-7477-45f1-967f-3f3444ffdaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:21.973446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dc1d4-7477-45f1-967f-3f3444ffdaac admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:21.973886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dc1d4-7477-45f1-967f-3f3444ffdaac admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:21.973999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc9bab-e130-4940-837e-e5cd3c032612 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:21.974407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dc1d4-7477-45f1-967f-3f3444ffdaac admin admin] PATH_INFO: `/v3/projects/874a97ef94334ee3af284946d2480fa7/users/e5999730ab88488db2071d3c696e67a6/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:21.987195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 969/1955] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:21.992536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc9bab-e130-4940-837e-e5cd3c032612 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.005126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dc1d4-7477-45f1-967f-3f3444ffdaac admin admin] RBAC: Authorizing `identity:create_grant(project_id=874a97ef94334ee3af284946d2480fa7, user_id=e5999730ab88488db2071d3c696e67a6, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.007659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc9bab-e130-4940-837e-e5cd3c032612 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.008260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dc1d4-7477-45f1-967f-3f3444ffdaac admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.016622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94295e74-ae9d-453e-8d6b-4003f5a6745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.017675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94295e74-ae9d-453e-8d6b-4003f5a6745a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.017844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94295e74-ae9d-453e-8d6b-4003f5a6745a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.018185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94295e74-ae9d-453e-8d6b-4003f5a6745a admin admin] PATH_INFO: `/v3/projects/5e68b78e3ae245a0bf612436f39aa76b/users/51108c968bba453da457f136a34f6527/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.031386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 987/1956] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:21 2022] PUT /identity/v3/projects/874a97ef94334ee3af284946d2480fa7/users/e5999730ab88488db2071d3c696e67a6/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:22.042330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccf681-dd90-4503-826e-4b4dba6884a2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.044687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccf681-dd90-4503-826e-4b4dba6884a2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.045472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdccf681-dd90-4503-826e-4b4dba6884a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.052431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94295e74-ae9d-453e-8d6b-4003f5a6745a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e68b78e3ae245a0bf612436f39aa76b, user_id=51108c968bba453da457f136a34f6527, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.055131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94295e74-ae9d-453e-8d6b-4003f5a6745a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.089772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-cdccf681-dd90-4503-826e-4b4dba6884a2 None None] MFA Rules not processed for user `e5999730ab88488db2071d3c696e67a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:22.091458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cdccf681-dd90-4503-826e-4b4dba6884a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:22.109786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 970/1957] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:21 2022] PUT /identity/v3/projects/5e68b78e3ae245a0bf612436f39aa76b/users/51108c968bba453da457f136a34f6527/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:22.120020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79af5f6-2759-4919-bf83-467a8f031aab None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.120668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79af5f6-2759-4919-bf83-467a8f031aab None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.120951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79af5f6-2759-4919-bf83-467a8f031aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.178146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f79af5f6-2759-4919-bf83-467a8f031aab None None] MFA Rules not processed for user `51108c968bba453da457f136a34f6527`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:22.179744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f79af5f6-2759-4919-bf83-467a8f031aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:22.196909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:22.200490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 988/1958] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:22.203097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccf681-dd90-4503-826e-4b4dba6884a2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.303816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:22.335811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdccf681-dd90-4503-826e-4b4dba6884a2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.343960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908f447-1624-4201-878e-2af7c5f17708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.344807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908f447-1624-4201-878e-2af7c5f17708 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.345802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908f447-1624-4201-878e-2af7c5f17708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.346432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908f447-1624-4201-878e-2af7c5f17708 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.347548 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908f447-1624-4201-878e-2af7c5f17708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.367909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 971/1959] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:22.367909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79af5f6-2759-4919-bf83-467a8f031aab None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.368181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908f447-1624-4201-878e-2af7c5f17708 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.378788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79af5f6-2759-4919-bf83-467a8f031aab None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.388411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed151de-0c3b-46f7-be4a-77af84cfea22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.389378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed151de-0c3b-46f7-be4a-77af84cfea22 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.389669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed151de-0c3b-46f7-be4a-77af84cfea22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.389941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed151de-0c3b-46f7-be4a-77af84cfea22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.390579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed151de-0c3b-46f7-be4a-77af84cfea22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.392531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed151de-0c3b-46f7-be4a-77af84cfea22 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.433348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 972/1960] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:22 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:22.438330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed151de-0c3b-46f7-be4a-77af84cfea22 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.439389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 989/1961] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:22 2022] POST /identity/v3/projects => generated 365 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:22.443086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908f447-1624-4201-878e-2af7c5f17708 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.526761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908f447-1624-4201-878e-2af7c5f17708 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.527287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed151de-0c3b-46f7-be4a-77af84cfea22 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.533429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6c1a2-b56e-460c-8f30-bc4f7488ded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.534221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6c1a2-b56e-460c-8f30-bc4f7488ded7 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.534775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6c1a2-b56e-460c-8f30-bc4f7488ded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.534775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6c1a2-b56e-460c-8f30-bc4f7488ded7 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.535515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6c1a2-b56e-460c-8f30-bc4f7488ded7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.537702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6c1a2-b56e-460c-8f30-bc4f7488ded7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.541119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fe65e-bfc0-4b42-b003-afcf50b93b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.541772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fe65e-bfc0-4b42-b003-afcf50b93b81 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.542449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fe65e-bfc0-4b42-b003-afcf50b93b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.542721 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fe65e-bfc0-4b42-b003-afcf50b93b81 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.543922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fe65e-bfc0-4b42-b003-afcf50b93b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.547606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fe65e-bfc0-4b42-b003-afcf50b93b81 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.575472 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 973/1962] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:22 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:22.576386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 990/1963] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:22 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:21:22.579332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fe65e-bfc0-4b42-b003-afcf50b93b81 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.582016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6c1a2-b56e-460c-8f30-bc4f7488ded7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.589404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fe65e-bfc0-4b42-b003-afcf50b93b81 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.593437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6c1a2-b56e-460c-8f30-bc4f7488ded7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.594333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dd7d8-827a-4eed-9b00-33b8c98e984f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.594931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dd7d8-827a-4eed-9b00-33b8c98e984f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.595206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dd7d8-827a-4eed-9b00-33b8c98e984f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.595472 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797dd7d8-827a-4eed-9b00-33b8c98e984f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.596138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dd7d8-827a-4eed-9b00-33b8c98e984f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.597715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797dd7d8-827a-4eed-9b00-33b8c98e984f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.598590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b916c-e850-4db0-8174-a0fef5aaa690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.599389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b916c-e850-4db0-8174-a0fef5aaa690 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.600035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b916c-e850-4db0-8174-a0fef5aaa690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.600035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b916c-e850-4db0-8174-a0fef5aaa690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.600608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b916c-e850-4db0-8174-a0fef5aaa690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.602244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b916c-e850-4db0-8174-a0fef5aaa690 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.608704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 974/1964] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:21:22.612803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dd7d8-827a-4eed-9b00-33b8c98e984f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.621500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 991/1965] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:21:22.624615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b916c-e850-4db0-8174-a0fef5aaa690 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.628941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797dd7d8-827a-4eed-9b00-33b8c98e984f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.633859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd0980-4865-4b8b-ae66-a893852c6e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.634515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd0980-4865-4b8b-ae66-a893852c6e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.634793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd0980-4865-4b8b-ae66-a893852c6e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.635059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd0980-4865-4b8b-ae66-a893852c6e16 admin admin] PATH_INFO: `/v3/projects/de2ae749b1004066a7f581d639191faf/users/516a72f40ce342e485bd6558c3405311/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.636974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b916c-e850-4db0-8174-a0fef5aaa690 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.642645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233a689-fe14-4d6d-8aa8-6ef31a356fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.643360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233a689-fe14-4d6d-8aa8-6ef31a356fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.643600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233a689-fe14-4d6d-8aa8-6ef31a356fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.643908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b233a689-fe14-4d6d-8aa8-6ef31a356fbc admin admin] PATH_INFO: `/v3/projects/7f4a866d003144b08fffc59395c5b8fd/users/09f0f387740e486dab2a5e4389529b09/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.651833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd0980-4865-4b8b-ae66-a893852c6e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2ae749b1004066a7f581d639191faf, user_id=516a72f40ce342e485bd6558c3405311, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.655003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd0980-4865-4b8b-ae66-a893852c6e16 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.659421 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233a689-fe14-4d6d-8aa8-6ef31a356fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4a866d003144b08fffc59395c5b8fd, user_id=09f0f387740e486dab2a5e4389529b09, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.662761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b233a689-fe14-4d6d-8aa8-6ef31a356fbc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.674518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 975/1966] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:22 2022] PUT /identity/v3/projects/de2ae749b1004066a7f581d639191faf/users/516a72f40ce342e485bd6558c3405311/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:21:22.679739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd0980-4865-4b8b-ae66-a893852c6e16 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.683294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 992/1967] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:22 2022] PUT /identity/v3/projects/7f4a866d003144b08fffc59395c5b8fd/users/09f0f387740e486dab2a5e4389529b09/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:21:22.687735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233a689-fe14-4d6d-8aa8-6ef31a356fbc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.735931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:22.754192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd0980-4865-4b8b-ae66-a893852c6e16 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.756213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b233a689-fe14-4d6d-8aa8-6ef31a356fbc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.760880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf455d93-1acf-4ad9-aec9-5f0f9d2de7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.761590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf455d93-1acf-4ad9-aec9-5f0f9d2de7ec admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.761839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97d7c2-2d32-419c-8223-be4903a2b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.761941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf455d93-1acf-4ad9-aec9-5f0f9d2de7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.762137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf455d93-1acf-4ad9-aec9-5f0f9d2de7ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.762645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97d7c2-2d32-419c-8223-be4903a2b903 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.762780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf455d93-1acf-4ad9-aec9-5f0f9d2de7ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.763084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97d7c2-2d32-419c-8223-be4903a2b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.763413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97d7c2-2d32-419c-8223-be4903a2b903 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.764221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97d7c2-2d32-419c-8223-be4903a2b903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.764534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf455d93-1acf-4ad9-aec9-5f0f9d2de7ec admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.766126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97d7c2-2d32-419c-8223-be4903a2b903 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.775296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 993/1968] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:21:22.778724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 976/1969] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:22.780208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf455d93-1acf-4ad9-aec9-5f0f9d2de7ec admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.780596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97d7c2-2d32-419c-8223-be4903a2b903 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.789919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf455d93-1acf-4ad9-aec9-5f0f9d2de7ec admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.790124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97d7c2-2d32-419c-8223-be4903a2b903 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:22.794477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36746234-6698-4f9c-b474-4e61a342c980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.794639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e0bab-74c9-4f20-81ba-1c344bdbbec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:22.795179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36746234-6698-4f9c-b474-4e61a342c980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.795955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e0bab-74c9-4f20-81ba-1c344bdbbec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.796060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36746234-6698-4f9c-b474-4e61a342c980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.796234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e0bab-74c9-4f20-81ba-1c344bdbbec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.796324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36746234-6698-4f9c-b474-4e61a342c980 admin admin] PATH_INFO: `/v3/projects/7f4a866d003144b08fffc59395c5b8fd/users/09f0f387740e486dab2a5e4389529b09/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.796507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e0bab-74c9-4f20-81ba-1c344bdbbec8 admin admin] PATH_INFO: `/v3/projects/de2ae749b1004066a7f581d639191faf/users/516a72f40ce342e485bd6558c3405311/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.798161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36746234-6698-4f9c-b474-4e61a342c980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4a866d003144b08fffc59395c5b8fd, user_id=09f0f387740e486dab2a5e4389529b09, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.798406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e0bab-74c9-4f20-81ba-1c344bdbbec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2ae749b1004066a7f581d639191faf, user_id=516a72f40ce342e485bd6558c3405311, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:22.800843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36746234-6698-4f9c-b474-4e61a342c980 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.801064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e0bab-74c9-4f20-81ba-1c344bdbbec8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:22.811603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 977/1970] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:22 2022] PUT /identity/v3/projects/de2ae749b1004066a7f581d639191faf/users/516a72f40ce342e485bd6558c3405311/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:21:22.820020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 994/1971] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:22 2022] PUT /identity/v3/projects/7f4a866d003144b08fffc59395c5b8fd/users/09f0f387740e486dab2a5e4389529b09/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:22.823138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f37f7a-c167-457c-907d-8476edabcecb None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.823420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f37f7a-c167-457c-907d-8476edabcecb None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.823691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f37f7a-c167-457c-907d-8476edabcecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.830993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c197a5-d39e-4539-9ff5-105a795b84d8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:22.831283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c197a5-d39e-4539-9ff5-105a795b84d8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:22.831554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c197a5-d39e-4539-9ff5-105a795b84d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:22.853769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-24f37f7a-c167-457c-907d-8476edabcecb None None] MFA Rules not processed for user `516a72f40ce342e485bd6558c3405311`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:22.854900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-24f37f7a-c167-457c-907d-8476edabcecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:22.859472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:22.862944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-84c197a5-d39e-4539-9ff5-105a795b84d8 None None] MFA Rules not processed for user `09f0f387740e486dab2a5e4389529b09`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:22.865074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-84c197a5-d39e-4539-9ff5-105a795b84d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:22.946056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 995/1972] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:22.948937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 978/1973] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:22.961007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c197a5-d39e-4539-9ff5-105a795b84d8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.962139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f37f7a-c167-457c-907d-8476edabcecb None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:22.962688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-84c197a5-d39e-4539-9ff5-105a795b84d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:22.963605 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-24f37f7a-c167-457c-907d-8476edabcecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:23.036757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c197a5-d39e-4539-9ff5-105a795b84d8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:23.040957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16356d-2a52-41de-8e0f-dc61b3c4dc8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:23.041094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f37f7a-c167-457c-907d-8476edabcecb None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:23.041791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16356d-2a52-41de-8e0f-dc61b3c4dc8e service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:23.042163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16356d-2a52-41de-8e0f-dc61b3c4dc8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:23.042530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16356d-2a52-41de-8e0f-dc61b3c4dc8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:23.044976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eaab6-2df7-4226-865d-86684a19cff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:23.046185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eaab6-2df7-4226-865d-86684a19cff2 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:23.046468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eaab6-2df7-4226-865d-86684a19cff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:23.046743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eaab6-2df7-4226-865d-86684a19cff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:23.057850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eaab6-2df7-4226-865d-86684a19cff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:23.059295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eaab6-2df7-4226-865d-86684a19cff2 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:23.071790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 979/1974] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:23.114078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16356d-2a52-41de-8e0f-dc61b3c4dc8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:23.115530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16356d-2a52-41de-8e0f-dc61b3c4dc8e service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:23.124701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 996/1975] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:23.157683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eaab6-2df7-4226-865d-86684a19cff2 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:23.175883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eaab6-2df7-4226-865d-86684a19cff2 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:23.179873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4ed70-e4e7-4012-b411-97524e7abedd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:23.180591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4ed70-e4e7-4012-b411-97524e7abedd service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:23.180979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4ed70-e4e7-4012-b411-97524e7abedd service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:23.181388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4ed70-e4e7-4012-b411-97524e7abedd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:23.281848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:23.307160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4ed70-e4e7-4012-b411-97524e7abedd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:23.308743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4ed70-e4e7-4012-b411-97524e7abedd service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:23.327986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 980/1976] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:29.272643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec3f63-73ec-4cfb-8b4f-732ac6094e5d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:29.273208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec3f63-73ec-4cfb-8b4f-732ac6094e5d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:29.273403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec3f63-73ec-4cfb-8b4f-732ac6094e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:29.296944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5bec3f63-73ec-4cfb-8b4f-732ac6094e5d None None] MFA Rules not processed for user `cd66d5d9989449729cec7e60cdaf0949`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:29.298284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5bec3f63-73ec-4cfb-8b4f-732ac6094e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:29.363535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 997/1977] 10.222.0.66 () {58 vars in 1015 bytes} [Wed Sep 21 12:21:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:29.438372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4ed70-e4e7-4012-b411-97524e7abedd service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:29.439877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7fa4ed70-e4e7-4012-b411-97524e7abedd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:29.470009 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4ed70-e4e7-4012-b411-97524e7abedd service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:29.474183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea736836-c39d-49b7-b2aa-60fa82699419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:29.474876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea736836-c39d-49b7-b2aa-60fa82699419 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:29.475233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea736836-c39d-49b7-b2aa-60fa82699419 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:29.475677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea736836-c39d-49b7-b2aa-60fa82699419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:29.484694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea736836-c39d-49b7-b2aa-60fa82699419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:29.486446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea736836-c39d-49b7-b2aa-60fa82699419 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:29.503747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 981/1978] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:29.508380 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec3f63-73ec-4cfb-8b4f-732ac6094e5d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:29.513556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5bec3f63-73ec-4cfb-8b4f-732ac6094e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:29.524505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:29.533825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec3f63-73ec-4cfb-8b4f-732ac6094e5d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:29.544820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8d664-98e2-4f4f-9927-8ddcb6b0bb8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:29.546216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8d664-98e2-4f4f-9927-8ddcb6b0bb8e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:29.550091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8d664-98e2-4f4f-9927-8ddcb6b0bb8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:29.550580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8d664-98e2-4f4f-9927-8ddcb6b0bb8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:29.570633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8d664-98e2-4f4f-9927-8ddcb6b0bb8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:29.578353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8d664-98e2-4f4f-9927-8ddcb6b0bb8e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:29.594588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 998/1979] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:30.876200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d52ed-edb4-4c88-b5f4-8df682183863 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:30.877044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d52ed-edb4-4c88-b5f4-8df682183863 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:30.877463 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d52ed-edb4-4c88-b5f4-8df682183863 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:30.879566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 982/1980] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:21:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:21:30.891555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521c472-e53c-406c-9d87-7bea6d0fa7a1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:30.892119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521c472-e53c-406c-9d87-7bea6d0fa7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:30.892119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b521c472-e53c-406c-9d87-7bea6d0fa7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:30.939083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b521c472-e53c-406c-9d87-7bea6d0fa7a1 None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:30.941568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b521c472-e53c-406c-9d87-7bea6d0fa7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:31.096319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 999/1981] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:21:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:31.116177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d52ed-edb4-4c88-b5f4-8df682183863 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:31.118158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b37d52ed-edb4-4c88-b5f4-8df682183863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:31.181536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d52ed-edb4-4c88-b5f4-8df682183863 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:31.203532 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d721fb-37a0-416f-9b0f-95dc5af37303 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:31.204406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d721fb-37a0-416f-9b0f-95dc5af37303 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:31.204751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d721fb-37a0-416f-9b0f-95dc5af37303 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:31.205046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d721fb-37a0-416f-9b0f-95dc5af37303 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:31.220372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d721fb-37a0-416f-9b0f-95dc5af37303 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:31.220664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d721fb-37a0-416f-9b0f-95dc5af37303 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:31.239066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 983/1982] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:31.852014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c381ac-4420-4406-9ac5-a65457bbf7cc None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:31.852310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c381ac-4420-4406-9ac5-a65457bbf7cc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:31.852593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c381ac-4420-4406-9ac5-a65457bbf7cc None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:31.854015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1000/1983] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:21:31.875887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8577c-511b-4eca-a1eb-affd3faea85f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:31.876175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8577c-511b-4eca-a1eb-affd3faea85f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:31.876446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8577c-511b-4eca-a1eb-affd3faea85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:31.897804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bff8577c-511b-4eca-a1eb-affd3faea85f None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:31.899126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bff8577c-511b-4eca-a1eb-affd3faea85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:31.909404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 984/1984] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:21:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:31.926343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c381ac-4420-4406-9ac5-a65457bbf7cc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:31.944160 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c381ac-4420-4406-9ac5-a65457bbf7cc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:31.952697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061250c-b98e-44c8-94a2-16d901d4a2de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:31.953713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061250c-b98e-44c8-94a2-16d901d4a2de service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:31.954198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061250c-b98e-44c8-94a2-16d901d4a2de service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:31.954683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061250c-b98e-44c8-94a2-16d901d4a2de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:31.965625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061250c-b98e-44c8-94a2-16d901d4a2de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:31.967735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061250c-b98e-44c8-94a2-16d901d4a2de service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:31.978754 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1001/1985] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:41.892544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=17 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:41.897870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8577c-511b-4eca-a1eb-affd3faea85f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:41.906710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8577c-511b-4eca-a1eb-affd3faea85f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:41.910223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3691ac-13bb-4a24-b966-5424733c7791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:41.910842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3691ac-13bb-4a24-b966-5424733c7791 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:41.911117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3691ac-13bb-4a24-b966-5424733c7791 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:41.911387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3691ac-13bb-4a24-b966-5424733c7791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:41.921262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3691ac-13bb-4a24-b966-5424733c7791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:41.922708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3691ac-13bb-4a24-b966-5424733c7791 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:41.932006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 985/1986] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:41.980216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=42 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:51.927777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:54.778244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a861a4-5ad6-4192-8d73-9bead4a80938 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:54.778771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a861a4-5ad6-4192-8d73-9bead4a80938 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:54.778771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a861a4-5ad6-4192-8d73-9bead4a80938 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:54.781435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1002/1987] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:21:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:21:54.793956 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9e258-1404-47f7-ba71-5a7a5f7ef185 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:54.794566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9e258-1404-47f7-ba71-5a7a5f7ef185 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:54.794566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9e258-1404-47f7-ba71-5a7a5f7ef185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:54.820763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-38c9e258-1404-47f7-ba71-5a7a5f7ef185 None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:54.820763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-38c9e258-1404-47f7-ba71-5a7a5f7ef185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:54.834568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 986/1988] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:21:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:54.908734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a861a4-5ad6-4192-8d73-9bead4a80938 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:54.911107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-88a861a4-5ad6-4192-8d73-9bead4a80938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:54.941010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a861a4-5ad6-4192-8d73-9bead4a80938 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:54.946626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14d7ac-a7aa-4fac-98e1-ed0c663c7be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:54.947278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14d7ac-a7aa-4fac-98e1-ed0c663c7be0 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:54.947560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14d7ac-a7aa-4fac-98e1-ed0c663c7be0 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:54.947867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14d7ac-a7aa-4fac-98e1-ed0c663c7be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:54.958309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14d7ac-a7aa-4fac-98e1-ed0c663c7be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:54.959953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14d7ac-a7aa-4fac-98e1-ed0c663c7be0 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:54.972834 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1003/1989] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:57.699533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9e258-1404-47f7-ba71-5a7a5f7ef185 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:57.793130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9e258-1404-47f7-ba71-5a7a5f7ef185 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:57.798612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:57.799322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:57.799736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:57.800040 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] PATH_INFO: `/v3/users/e5999730ab88488db2071d3c696e67a6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:57.801228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5999730ab88488db2071d3c696e67a6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:57.803191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:57.868581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5999730ab88488db2071d3c696e67a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:57.869485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5999730ab88488db2071d3c696e67a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:57.878997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5999730ab88488db2071d3c696e67a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:57.883433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5999730ab88488db2071d3c696e67a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:57.891117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5999730ab88488db2071d3c696e67a6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:57.915330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 987/1990] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:57 2022] DELETE /identity/v3/users/e5999730ab88488db2071d3c696e67a6 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:57.919890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832eac-2909-41a7-a116-bde9136177fd None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:57.921019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832eac-2909-41a7-a116-bde9136177fd None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:57.921019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18832eac-2909-41a7-a116-bde9136177fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:57.941283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-18832eac-2909-41a7-a116-bde9136177fd None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:57.942601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-18832eac-2909-41a7-a116-bde9136177fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:58.007708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1004/1991] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:58.017290 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:58.018802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:58.083177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261831c6-032a-4d82-9931-a94b1b83f9ca admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:58.088124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117968ed-8a06-431e-949f-2a952fd46fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:58.088747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117968ed-8a06-431e-949f-2a952fd46fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:58.089031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117968ed-8a06-431e-949f-2a952fd46fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:58.089308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117968ed-8a06-431e-949f-2a952fd46fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:58.099312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117968ed-8a06-431e-949f-2a952fd46fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:58.100779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117968ed-8a06-431e-949f-2a952fd46fa4 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:58.112704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 988/1992] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:21:58.568469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832eac-2909-41a7-a116-bde9136177fd None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:58.570706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-18832eac-2909-41a7-a116-bde9136177fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:58.585936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18832eac-2909-41a7-a116-bde9136177fd None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:58.592706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:58.603918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:58.604474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:58.604761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] PATH_INFO: `/v3/projects/874a97ef94334ee3af284946d2480fa7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:58.606180 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=874a97ef94334ee3af284946d2480fa7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:58.608243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:58.660822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '874a97ef94334ee3af284946d2480fa7', 'request_id': 'req-3eeba04e-97ea-42b7-81d6-55cce96cdc20'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.668446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '874a97ef94334ee3af284946d2480fa7', 'request_id': 'req-3eeba04e-97ea-42b7-81d6-55cce96cdc20'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.678175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '874a97ef94334ee3af284946d2480fa7', 'request_id': 'req-3eeba04e-97ea-42b7-81d6-55cce96cdc20'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.696751 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] The token cache is being invalidate because project 874a97ef94334ee3af284946d2480fa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:58.697083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.697944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.699617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1005/1993] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:58 2022] DELETE /identity/v3/projects/874a97ef94334ee3af284946d2480fa7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:58.704411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117968ed-8a06-431e-949f-2a952fd46fa4 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:58.715299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117968ed-8a06-431e-949f-2a952fd46fa4 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:58.727768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:58.780907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:58.781542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:58.781824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:58.782099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] PATH_INFO: `/v3/users/09f0f387740e486dab2a5e4389529b09` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:58.783225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09f0f387740e486dab2a5e4389529b09)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:58.785199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:58.835796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09f0f387740e486dab2a5e4389529b09'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.836743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09f0f387740e486dab2a5e4389529b09'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.845490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09f0f387740e486dab2a5e4389529b09'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.846671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09f0f387740e486dab2a5e4389529b09'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.855291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-c0c3eeda-334d-4136-84cc-307fd7b92899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09f0f387740e486dab2a5e4389529b09'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:58.879077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 989/1994] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:21:58 2022] DELETE /identity/v3/users/09f0f387740e486dab2a5e4389529b09 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:59.149966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:59.152013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:59.170890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeba04e-97ea-42b7-81d6-55cce96cdc20 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:59.233355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:59.234002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.234281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.234554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] PATH_INFO: `/v3/projects/7f4a866d003144b08fffc59395c5b8fd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.235770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f4a866d003144b08fffc59395c5b8fd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:59.237714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:59.251015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:59.286690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f4a866d003144b08fffc59395c5b8fd', 'request_id': 'req-2525c850-58fb-42a9-bbbf-0e4e2c878754'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:59.292719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f4a866d003144b08fffc59395c5b8fd', 'request_id': 'req-2525c850-58fb-42a9-bbbf-0e4e2c878754'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:59.300602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f4a866d003144b08fffc59395c5b8fd', 'request_id': 'req-2525c850-58fb-42a9-bbbf-0e4e2c878754'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:59.311327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] The token cache is being invalidate because project 7f4a866d003144b08fffc59395c5b8fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:21:59.311615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:59.312275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2525c850-58fb-42a9-bbbf-0e4e2c878754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:21:59.313786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1006/1995] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:21:59 2022] DELETE /identity/v3/projects/7f4a866d003144b08fffc59395c5b8fd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:59.318546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44189fdb-36d6-4412-ae12-016cd863daa2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.318840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44189fdb-36d6-4412-ae12-016cd863daa2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.319145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44189fdb-36d6-4412-ae12-016cd863daa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.339641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-44189fdb-36d6-4412-ae12-016cd863daa2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:59.340784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-44189fdb-36d6-4412-ae12-016cd863daa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:59.398699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 990/1996] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:59.406863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ff017-1ea0-44b1-bfc1-4c59c9191d2c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.407144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ff017-1ea0-44b1-bfc1-4c59c9191d2c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.407415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ff017-1ea0-44b1-bfc1-4c59c9191d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.427204 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-687ff017-1ea0-44b1-bfc1-4c59c9191d2c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:59.428448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-687ff017-1ea0-44b1-bfc1-4c59c9191d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:59.436203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1007/1997] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:21:59.440409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44189fdb-36d6-4412-ae12-016cd863daa2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:59.450482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44189fdb-36d6-4412-ae12-016cd863daa2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:59.455214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e8a3-726e-4b57-93a2-1d9af235d630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:59.455873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e8a3-726e-4b57-93a2-1d9af235d630 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.456142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e8a3-726e-4b57-93a2-1d9af235d630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.456409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51e8a3-726e-4b57-93a2-1d9af235d630 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.457339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51e8a3-726e-4b57-93a2-1d9af235d630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:21:59.457650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51e8a3-726e-4b57-93a2-1d9af235d630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:59.459207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b51e8a3-726e-4b57-93a2-1d9af235d630 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:59.474111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 991/1998] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:21:59.476113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ff017-1ea0-44b1-bfc1-4c59c9191d2c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:59.485768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ff017-1ea0-44b1-bfc1-4c59c9191d2c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:59.490391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5e17f-472b-432a-8e3a-81c4215d6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:59.491212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5e17f-472b-432a-8e3a-81c4215d6591 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.491634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5e17f-472b-432a-8e3a-81c4215d6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.492098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5e17f-472b-432a-8e3a-81c4215d6591 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.493067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f5e17f-472b-432a-8e3a-81c4215d6591 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:59.494786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f5e17f-472b-432a-8e3a-81c4215d6591 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:59.523232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1008/1999] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:59 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:59.528063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e8a3-726e-4b57-93a2-1d9af235d630 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:59.541469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51e8a3-726e-4b57-93a2-1d9af235d630 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:59.570534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:21:59.615240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ace58-0059-494d-9793-40ff6cc6b191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:59.615883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ace58-0059-494d-9793-40ff6cc6b191 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.616184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ace58-0059-494d-9793-40ff6cc6b191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.616445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ace58-0059-494d-9793-40ff6cc6b191 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.617073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ace58-0059-494d-9793-40ff6cc6b191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:59.619007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ace58-0059-494d-9793-40ff6cc6b191 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:59.641208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 992/2000] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:21:59 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:21:59.645944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5e17f-472b-432a-8e3a-81c4215d6591 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:59.656908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5e17f-472b-432a-8e3a-81c4215d6591 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:59.661482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91257cbd-3043-4365-bdcd-a6b6cbfe64bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:59.662107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91257cbd-3043-4365-bdcd-a6b6cbfe64bb admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.662388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91257cbd-3043-4365-bdcd-a6b6cbfe64bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.662661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91257cbd-3043-4365-bdcd-a6b6cbfe64bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.663296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91257cbd-3043-4365-bdcd-a6b6cbfe64bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:59.664912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91257cbd-3043-4365-bdcd-a6b6cbfe64bb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:59.675370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1009/2001] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:59.679334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ace58-0059-494d-9793-40ff6cc6b191 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:59.689349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ace58-0059-494d-9793-40ff6cc6b191 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:59.694021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbc647-cae8-4940-a738-e4b8f989c6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:59.694620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbc647-cae8-4940-a738-e4b8f989c6bd admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.694901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbc647-cae8-4940-a738-e4b8f989c6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.695176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbc647-cae8-4940-a738-e4b8f989c6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.695850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbc647-cae8-4940-a738-e4b8f989c6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:59.697443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcbc647-cae8-4940-a738-e4b8f989c6bd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:59.708945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 993/2002] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:21:59.712890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91257cbd-3043-4365-bdcd-a6b6cbfe64bb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:59.729751 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91257cbd-3043-4365-bdcd-a6b6cbfe64bb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:59.735394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14e532-5d4f-40f4-948b-1e024629fc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:59.736096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14e532-5d4f-40f4-948b-1e024629fc86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.736378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14e532-5d4f-40f4-948b-1e024629fc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.736664 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14e532-5d4f-40f4-948b-1e024629fc86 admin admin] PATH_INFO: `/v3/projects/b249fcd800ff4111be8e234fce1bf6ac/users/e3ca35354a704ff0adb29f2e6fb67e27/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.753531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14e532-5d4f-40f4-948b-1e024629fc86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b249fcd800ff4111be8e234fce1bf6ac, user_id=e3ca35354a704ff0adb29f2e6fb67e27, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:59.757062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14e532-5d4f-40f4-948b-1e024629fc86 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:21:59.782650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1010/2003] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:21:59 2022] PUT /identity/v3/projects/b249fcd800ff4111be8e234fce1bf6ac/users/e3ca35354a704ff0adb29f2e6fb67e27/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:21:59.793943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cc112-3018-4d2e-9b49-089409d22ac0 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.794237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cc112-3018-4d2e-9b49-089409d22ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.794514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cc112-3018-4d2e-9b49-089409d22ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.822386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-243cc112-3018-4d2e-9b49-089409d22ac0 None None] MFA Rules not processed for user `e3ca35354a704ff0adb29f2e6fb67e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:21:59.823604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-243cc112-3018-4d2e-9b49-089409d22ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:21:59.894621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 994/2004] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:21:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:21:59.898867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14e532-5d4f-40f4-948b-1e024629fc86 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:21:59.972264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14e532-5d4f-40f4-948b-1e024629fc86 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:21:59.977405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899910e-a756-470e-bf1c-193f85353833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:21:59.978193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899910e-a756-470e-bf1c-193f85353833 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:21:59.978499 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899910e-a756-470e-bf1c-193f85353833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:21:59.978810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899910e-a756-470e-bf1c-193f85353833 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:21:59.979454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899910e-a756-470e-bf1c-193f85353833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:21:59.981398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9899910e-a756-470e-bf1c-193f85353833 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:00.008753 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1011/2005] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:21:59 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:00.012783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cc112-3018-4d2e-9b49-089409d22ac0 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:00.023891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cc112-3018-4d2e-9b49-089409d22ac0 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:00.087482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74648fe9-6a1c-4a43-a344-70932062b02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:00.088127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74648fe9-6a1c-4a43-a344-70932062b02a admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:00.088404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74648fe9-6a1c-4a43-a344-70932062b02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:00.088672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74648fe9-6a1c-4a43-a344-70932062b02a admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:00.089288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74648fe9-6a1c-4a43-a344-70932062b02a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:00.091176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74648fe9-6a1c-4a43-a344-70932062b02a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:00.113107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 995/2006] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:00 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:22:00.117177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899910e-a756-470e-bf1c-193f85353833 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:00.127831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899910e-a756-470e-bf1c-193f85353833 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:00.132421 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707845c-e0bc-41e5-bfb3-8a142dd02020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:00.133024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707845c-e0bc-41e5-bfb3-8a142dd02020 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:00.133302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707845c-e0bc-41e5-bfb3-8a142dd02020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:00.133578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707845c-e0bc-41e5-bfb3-8a142dd02020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:00.134212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707845c-e0bc-41e5-bfb3-8a142dd02020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:00.135835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707845c-e0bc-41e5-bfb3-8a142dd02020 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:00.146525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1012/2007] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:00.150873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74648fe9-6a1c-4a43-a344-70932062b02a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:00.161253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74648fe9-6a1c-4a43-a344-70932062b02a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:00.166123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397407a3-8b7d-4bc5-9928-e1f1f54f7b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:00.166771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397407a3-8b7d-4bc5-9928-e1f1f54f7b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:00.167050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397407a3-8b7d-4bc5-9928-e1f1f54f7b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:00.167324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397407a3-8b7d-4bc5-9928-e1f1f54f7b7d admin admin] PATH_INFO: `/v3/projects/c83b0a987d224c9daed0e8ff094810e4/users/8d3ed4614a1843e2be1135b7f80a8a70/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:00.183713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397407a3-8b7d-4bc5-9928-e1f1f54f7b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83b0a987d224c9daed0e8ff094810e4, user_id=8d3ed4614a1843e2be1135b7f80a8a70, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:00.186446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397407a3-8b7d-4bc5-9928-e1f1f54f7b7d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:00.189704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:00.208484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 996/2008] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:00 2022] PUT /identity/v3/projects/c83b0a987d224c9daed0e8ff094810e4/users/8d3ed4614a1843e2be1135b7f80a8a70/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:00.212364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707845c-e0bc-41e5-bfb3-8a142dd02020 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:00.223331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707845c-e0bc-41e5-bfb3-8a142dd02020 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:00.231320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:00.285074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d6f57-b689-4a6e-9095-8fe6108c1f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:00.285686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d6f57-b689-4a6e-9095-8fe6108c1f4e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:00.285963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d6f57-b689-4a6e-9095-8fe6108c1f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:00.286239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d6f57-b689-4a6e-9095-8fe6108c1f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:00.286871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d6f57-b689-4a6e-9095-8fe6108c1f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:00.288504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d6f57-b689-4a6e-9095-8fe6108c1f4e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:00.298761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1013/2009] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:00.303076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397407a3-8b7d-4bc5-9928-e1f1f54f7b7d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:00.313283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397407a3-8b7d-4bc5-9928-e1f1f54f7b7d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:00.317967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2c6f1-32a8-4424-8e56-1794e5acb286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:00.318609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2c6f1-32a8-4424-8e56-1794e5acb286 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:00.318886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2c6f1-32a8-4424-8e56-1794e5acb286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:00.319156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a2c6f1-32a8-4424-8e56-1794e5acb286 admin admin] PATH_INFO: `/v3/projects/c83b0a987d224c9daed0e8ff094810e4/users/8d3ed4614a1843e2be1135b7f80a8a70/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:00.321094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2c6f1-32a8-4424-8e56-1794e5acb286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83b0a987d224c9daed0e8ff094810e4, user_id=8d3ed4614a1843e2be1135b7f80a8a70, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:00.324049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a2c6f1-32a8-4424-8e56-1794e5acb286 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:00.334061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 997/2010] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:00 2022] PUT /identity/v3/projects/c83b0a987d224c9daed0e8ff094810e4/users/8d3ed4614a1843e2be1135b7f80a8a70/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:00.344691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891201c8-9c26-43ed-a025-5dd02f36d3e9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:00.344982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891201c8-9c26-43ed-a025-5dd02f36d3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:00.345260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891201c8-9c26-43ed-a025-5dd02f36d3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:00.373308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-891201c8-9c26-43ed-a025-5dd02f36d3e9 None None] MFA Rules not processed for user `8d3ed4614a1843e2be1135b7f80a8a70`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:00.374521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-891201c8-9c26-43ed-a025-5dd02f36d3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:00.442247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1014/2011] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:00.454000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2c6f1-32a8-4424-8e56-1794e5acb286 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:00.456317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2a2c6f1-32a8-4424-8e56-1794e5acb286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:00.513457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a2c6f1-32a8-4424-8e56-1794e5acb286 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:00.517517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd93d0-8486-44d3-9902-6970ad35d24a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:00.518113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd93d0-8486-44d3-9902-6970ad35d24a service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:00.518388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd93d0-8486-44d3-9902-6970ad35d24a service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:00.518656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd93d0-8486-44d3-9902-6970ad35d24a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:00.527423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd93d0-8486-44d3-9902-6970ad35d24a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:00.528998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd93d0-8486-44d3-9902-6970ad35d24a service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:00.539165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 998/2012] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:00.643239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891201c8-9c26-43ed-a025-5dd02f36d3e9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:00.644915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-891201c8-9c26-43ed-a025-5dd02f36d3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:00.657933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891201c8-9c26-43ed-a025-5dd02f36d3e9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:00.661674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecef89-fa61-4de6-9334-34eb3c400f2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:00.662376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecef89-fa61-4de6-9334-34eb3c400f2a service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:00.662659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecef89-fa61-4de6-9334-34eb3c400f2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:00.662930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecef89-fa61-4de6-9334-34eb3c400f2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:00.730215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecef89-fa61-4de6-9334-34eb3c400f2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:00.731682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecef89-fa61-4de6-9334-34eb3c400f2a service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:00.745730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1015/2013] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:03.603698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd93d0-8486-44d3-9902-6970ad35d24a service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:03.666719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd93d0-8486-44d3-9902-6970ad35d24a service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:03.671223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:03.671888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:03.672171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:03.672444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] PATH_INFO: `/v3/users/e3ca35354a704ff0adb29f2e6fb67e27` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:03.673564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3ca35354a704ff0adb29f2e6fb67e27)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:03.675479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:03.722751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3ca35354a704ff0adb29f2e6fb67e27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:03.723406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3ca35354a704ff0adb29f2e6fb67e27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:03.730764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3ca35354a704ff0adb29f2e6fb67e27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:03.731553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3ca35354a704ff0adb29f2e6fb67e27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:03.739941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3ca35354a704ff0adb29f2e6fb67e27'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:03.759714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 999/2014] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:03 2022] DELETE /identity/v3/users/e3ca35354a704ff0adb29f2e6fb67e27 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:03.768708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ffcce-befb-4fea-a491-569c7a1ef1e0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:03.769103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ffcce-befb-4fea-a491-569c7a1ef1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:03.769347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ffcce-befb-4fea-a491-569c7a1ef1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:03.793715 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c39ffcce-befb-4fea-a491-569c7a1ef1e0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:03.795049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c39ffcce-befb-4fea-a491-569c7a1ef1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:03.820445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:03.855672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1016/2015] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:03.865901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:03.867366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:03.874603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:03.927868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f665dbd-0e96-4d61-8cf6-afdf5bf6d672 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:03.931672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa2382-79d3-4462-b8b9-f17abdbb20f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:03.932743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa2382-79d3-4462-b8b9-f17abdbb20f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:03.932743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa2382-79d3-4462-b8b9-f17abdbb20f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:03.932955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa2382-79d3-4462-b8b9-f17abdbb20f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:03.943968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa2382-79d3-4462-b8b9-f17abdbb20f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:03.945407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa2382-79d3-4462-b8b9-f17abdbb20f3 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:03.955731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1000/2016] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:04.251787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ffcce-befb-4fea-a491-569c7a1ef1e0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:04.253262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c39ffcce-befb-4fea-a491-569c7a1ef1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:04.265459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ffcce-befb-4fea-a491-569c7a1ef1e0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:04.281165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:04.281850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:04.282125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:04.282413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] PATH_INFO: `/v3/projects/b249fcd800ff4111be8e234fce1bf6ac` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:04.283659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b249fcd800ff4111be8e234fce1bf6ac)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:04.285662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:04.356000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b249fcd800ff4111be8e234fce1bf6ac', 'request_id': 'req-c9435bf9-61f0-433d-a722-7e8e213c6199'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.356000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b249fcd800ff4111be8e234fce1bf6ac', 'request_id': 'req-c9435bf9-61f0-433d-a722-7e8e213c6199'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.360257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b249fcd800ff4111be8e234fce1bf6ac', 'request_id': 'req-c9435bf9-61f0-433d-a722-7e8e213c6199'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.370608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] The token cache is being invalidate because project b249fcd800ff4111be8e234fce1bf6ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:04.370908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.371500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.373600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1017/2017] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:04 2022] DELETE /identity/v3/projects/b249fcd800ff4111be8e234fce1bf6ac => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:04.377820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa2382-79d3-4462-b8b9-f17abdbb20f3 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:04.387397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa2382-79d3-4462-b8b9-f17abdbb20f3 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:04.443590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:04.444244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:04.444524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:04.444796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] PATH_INFO: `/v3/users/8d3ed4614a1843e2be1135b7f80a8a70` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:04.445909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d3ed4614a1843e2be1135b7f80a8a70)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:04.447893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:04.495094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d3ed4614a1843e2be1135b7f80a8a70'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.495739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d3ed4614a1843e2be1135b7f80a8a70'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.503435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d3ed4614a1843e2be1135b7f80a8a70'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.504070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d3ed4614a1843e2be1135b7f80a8a70'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.511106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-ac400948-4915-49e1-a1b4-93c7f3442565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d3ed4614a1843e2be1135b7f80a8a70'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:04.527996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1001/2018] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:04 2022] DELETE /identity/v3/users/8d3ed4614a1843e2be1135b7f80a8a70 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:04.879420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:04.880932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:04.893957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9435bf9-61f0-433d-a722-7e8e213c6199 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:04.963875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:04.964511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:04.964793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:04.965069 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] PATH_INFO: `/v3/projects/c83b0a987d224c9daed0e8ff094810e4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:04.966188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83b0a987d224c9daed0e8ff094810e4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:04.968758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.014805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83b0a987d224c9daed0e8ff094810e4', 'request_id': 'req-1f313900-e5d5-48df-9d49-53293e0ad80e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:05.020881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83b0a987d224c9daed0e8ff094810e4', 'request_id': 'req-1f313900-e5d5-48df-9d49-53293e0ad80e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:05.023626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:05.031683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83b0a987d224c9daed0e8ff094810e4', 'request_id': 'req-1f313900-e5d5-48df-9d49-53293e0ad80e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:05.044094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] The token cache is being invalidate because project c83b0a987d224c9daed0e8ff094810e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:05.044383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:05.045609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1f313900-e5d5-48df-9d49-53293e0ad80e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:05.047377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1018/2019] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:04 2022] DELETE /identity/v3/projects/c83b0a987d224c9daed0e8ff094810e4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:05.052882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e4f38-fa9a-4bf9-97bf-15968d128d12 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.053405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e4f38-fa9a-4bf9-97bf-15968d128d12 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.053405 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e4f38-fa9a-4bf9-97bf-15968d128d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.073595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ff6e4f38-fa9a-4bf9-97bf-15968d128d12 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:05.074805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ff6e4f38-fa9a-4bf9-97bf-15968d128d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:05.095144 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:05.137545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1002/2020] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:05.146561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467fb14-f129-49f8-acb8-2f0a2db21293 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.146842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467fb14-f129-49f8-acb8-2f0a2db21293 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.147114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467fb14-f129-49f8-acb8-2f0a2db21293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.167218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1467fb14-f129-49f8-acb8-2f0a2db21293 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:05.168452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1467fb14-f129-49f8-acb8-2f0a2db21293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:05.176903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1019/2021] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:05.181201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e4f38-fa9a-4bf9-97bf-15968d128d12 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.191536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e4f38-fa9a-4bf9-97bf-15968d128d12 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.196077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dce57-aa13-44f2-b054-ea95b54af1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.196676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dce57-aa13-44f2-b054-ea95b54af1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.196955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dce57-aa13-44f2-b054-ea95b54af1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.197229 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dce57-aa13-44f2-b054-ea95b54af1d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.197870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dce57-aa13-44f2-b054-ea95b54af1d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:05.198172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dce57-aa13-44f2-b054-ea95b54af1d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.199791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dce57-aa13-44f2-b054-ea95b54af1d1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.211383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1003/2022] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:22:05.215698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467fb14-f129-49f8-acb8-2f0a2db21293 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.225015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467fb14-f129-49f8-acb8-2f0a2db21293 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.229696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25691c-20ed-480b-95d2-9edb58b03d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.230296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25691c-20ed-480b-95d2-9edb58b03d41 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.230577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25691c-20ed-480b-95d2-9edb58b03d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.230849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25691c-20ed-480b-95d2-9edb58b03d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.231490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25691c-20ed-480b-95d2-9edb58b03d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.233929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25691c-20ed-480b-95d2-9edb58b03d41 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.261345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1020/2023] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:05 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:05.265719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dce57-aa13-44f2-b054-ea95b54af1d1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.275994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dce57-aa13-44f2-b054-ea95b54af1d1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.335036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1579402-d489-4d5f-b52f-fbade1376e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.335551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1579402-d489-4d5f-b52f-fbade1376e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.335864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1579402-d489-4d5f-b52f-fbade1376e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.336134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1579402-d489-4d5f-b52f-fbade1376e6b admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.336763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1579402-d489-4d5f-b52f-fbade1376e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.338672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1579402-d489-4d5f-b52f-fbade1376e6b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.364079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1004/2024] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:05 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:05.365476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25691c-20ed-480b-95d2-9edb58b03d41 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.376351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25691c-20ed-480b-95d2-9edb58b03d41 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.381103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df5d7e-cb4e-4ea0-a631-8d72abd357fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.381736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df5d7e-cb4e-4ea0-a631-8d72abd357fd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.382021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df5d7e-cb4e-4ea0-a631-8d72abd357fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.382296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df5d7e-cb4e-4ea0-a631-8d72abd357fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.382940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df5d7e-cb4e-4ea0-a631-8d72abd357fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.384591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df5d7e-cb4e-4ea0-a631-8d72abd357fd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.394636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1021/2025] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:05.398686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1579402-d489-4d5f-b52f-fbade1376e6b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.409210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1579402-d489-4d5f-b52f-fbade1376e6b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.414000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437da22-bce3-4eb4-95e2-38381eb38de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.414609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437da22-bce3-4eb4-95e2-38381eb38de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.414889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437da22-bce3-4eb4-95e2-38381eb38de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.415161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437da22-bce3-4eb4-95e2-38381eb38de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.415835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437da22-bce3-4eb4-95e2-38381eb38de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.417446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437da22-bce3-4eb4-95e2-38381eb38de1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.431068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1005/2026] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:05.435235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df5d7e-cb4e-4ea0-a631-8d72abd357fd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.445647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df5d7e-cb4e-4ea0-a631-8d72abd357fd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.450541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17a0ce-a466-4cbc-8e3e-d196b904ac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.451253 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17a0ce-a466-4cbc-8e3e-d196b904ac59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.451538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17a0ce-a466-4cbc-8e3e-d196b904ac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.451842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17a0ce-a466-4cbc-8e3e-d196b904ac59 admin admin] PATH_INFO: `/v3/projects/a9f7810d7b2046a1a574cadd97a35f86/users/6d965e6045594412b6614ab697ead08c/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.470691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17a0ce-a466-4cbc-8e3e-d196b904ac59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f7810d7b2046a1a574cadd97a35f86, user_id=6d965e6045594412b6614ab697ead08c, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.473893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17a0ce-a466-4cbc-8e3e-d196b904ac59 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.500674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1022/2027] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:05 2022] PUT /identity/v3/projects/a9f7810d7b2046a1a574cadd97a35f86/users/6d965e6045594412b6614ab697ead08c/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:05.511381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419e9a-9538-4fa2-a0ae-c0c7a3c29509 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.511674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419e9a-9538-4fa2-a0ae-c0c7a3c29509 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.511985 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da419e9a-9538-4fa2-a0ae-c0c7a3c29509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.540792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-da419e9a-9538-4fa2-a0ae-c0c7a3c29509 None None] MFA Rules not processed for user `6d965e6045594412b6614ab697ead08c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:05.542002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-da419e9a-9538-4fa2-a0ae-c0c7a3c29509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:05.608574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1006/2028] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:05.613290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17a0ce-a466-4cbc-8e3e-d196b904ac59 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.673555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17a0ce-a466-4cbc-8e3e-d196b904ac59 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.678155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd87a74-fe90-43e1-8817-dfd254736202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.678749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd87a74-fe90-43e1-8817-dfd254736202 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.679029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd87a74-fe90-43e1-8817-dfd254736202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.679294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd87a74-fe90-43e1-8817-dfd254736202 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.679938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd87a74-fe90-43e1-8817-dfd254736202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.681828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd87a74-fe90-43e1-8817-dfd254736202 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.707960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1023/2029] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:05 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:05.712023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da419e9a-9538-4fa2-a0ae-c0c7a3c29509 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.723193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da419e9a-9538-4fa2-a0ae-c0c7a3c29509 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.740835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:05.788133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29492da-9bd0-410e-b14b-f97459500f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.788749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29492da-9bd0-410e-b14b-f97459500f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.789031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29492da-9bd0-410e-b14b-f97459500f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.789307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29492da-9bd0-410e-b14b-f97459500f29 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.789952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29492da-9bd0-410e-b14b-f97459500f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.791895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29492da-9bd0-410e-b14b-f97459500f29 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.817065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1007/2030] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:05 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:05.819033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd87a74-fe90-43e1-8817-dfd254736202 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.830342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd87a74-fe90-43e1-8817-dfd254736202 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.835304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f34a0e-07b4-44e5-8fc2-d8fd7d722545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.835971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f34a0e-07b4-44e5-8fc2-d8fd7d722545 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.836258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f34a0e-07b4-44e5-8fc2-d8fd7d722545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.836534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f34a0e-07b4-44e5-8fc2-d8fd7d722545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.837231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f34a0e-07b4-44e5-8fc2-d8fd7d722545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.838799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f34a0e-07b4-44e5-8fc2-d8fd7d722545 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.851375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1024/2031] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:05.855664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29492da-9bd0-410e-b14b-f97459500f29 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.870455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29492da-9bd0-410e-b14b-f97459500f29 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.877294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a16ec-e10e-4ce8-b43a-faf50287e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.878352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a16ec-e10e-4ce8-b43a-faf50287e8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.879522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a16ec-e10e-4ce8-b43a-faf50287e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.879950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a16ec-e10e-4ce8-b43a-faf50287e8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.880901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a16ec-e10e-4ce8-b43a-faf50287e8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.883419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176a16ec-e10e-4ce8-b43a-faf50287e8d5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.898507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1008/2032] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:05.905648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f34a0e-07b4-44e5-8fc2-d8fd7d722545 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:05.924496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f34a0e-07b4-44e5-8fc2-d8fd7d722545 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:05.931313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819d21f-557d-46a3-abbc-90d89b19e1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:05.932174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819d21f-557d-46a3-abbc-90d89b19e1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.932559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819d21f-557d-46a3-abbc-90d89b19e1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.932930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819d21f-557d-46a3-abbc-90d89b19e1b0 admin admin] PATH_INFO: `/v3/projects/055fefe35dde4a8eb2752797e80a34cd/users/5d78ca2180d34ad59beb3e361cd8e4bb/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:05.950101 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819d21f-557d-46a3-abbc-90d89b19e1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055fefe35dde4a8eb2752797e80a34cd, user_id=5d78ca2180d34ad59beb3e361cd8e4bb, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:05.953605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5819d21f-557d-46a3-abbc-90d89b19e1b0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:05.967927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1025/2033] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:05 2022] PUT /identity/v3/projects/055fefe35dde4a8eb2752797e80a34cd/users/5d78ca2180d34ad59beb3e361cd8e4bb/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:05.978063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5de2-72a5-47cc-8446-738eb3d073ad None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:05.978419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5de2-72a5-47cc-8446-738eb3d073ad None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:05.978762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e5de2-72a5-47cc-8446-738eb3d073ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:06.006964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ed1e5de2-72a5-47cc-8446-738eb3d073ad None None] MFA Rules not processed for user `5d78ca2180d34ad59beb3e361cd8e4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:06.008314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ed1e5de2-72a5-47cc-8446-738eb3d073ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:06.072769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1009/2034] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:06.077708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819d21f-557d-46a3-abbc-90d89b19e1b0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:06.113034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:06.156205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819d21f-557d-46a3-abbc-90d89b19e1b0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:06.160772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605b5b2-799b-429f-88ff-10ac1e0af5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:06.161372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605b5b2-799b-429f-88ff-10ac1e0af5db admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:06.161650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605b5b2-799b-429f-88ff-10ac1e0af5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:06.161935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605b5b2-799b-429f-88ff-10ac1e0af5db admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:06.162562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605b5b2-799b-429f-88ff-10ac1e0af5db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:06.164479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605b5b2-799b-429f-88ff-10ac1e0af5db admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:06.190995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1026/2035] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:06 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:06.195010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5de2-72a5-47cc-8446-738eb3d073ad None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:06.206360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e5de2-72a5-47cc-8446-738eb3d073ad None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:06.262901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4058f05-de8e-487d-a600-00bae1da590f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:06.264137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4058f05-de8e-487d-a600-00bae1da590f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:06.264426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4058f05-de8e-487d-a600-00bae1da590f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:06.264698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4058f05-de8e-487d-a600-00bae1da590f admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:06.265333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4058f05-de8e-487d-a600-00bae1da590f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:06.267265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4058f05-de8e-487d-a600-00bae1da590f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:06.288556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1010/2036] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:06 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:06.292553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605b5b2-799b-429f-88ff-10ac1e0af5db admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:06.302941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605b5b2-799b-429f-88ff-10ac1e0af5db admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:06.307561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf602e-cbb5-4741-9bdd-cdc2a044a481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:06.308191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf602e-cbb5-4741-9bdd-cdc2a044a481 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:06.308471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf602e-cbb5-4741-9bdd-cdc2a044a481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:06.308748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf602e-cbb5-4741-9bdd-cdc2a044a481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:06.309381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf602e-cbb5-4741-9bdd-cdc2a044a481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:06.311500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf602e-cbb5-4741-9bdd-cdc2a044a481 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:06.322129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1027/2037] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:06.326725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4058f05-de8e-487d-a600-00bae1da590f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:06.337588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4058f05-de8e-487d-a600-00bae1da590f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:06.342418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc0bf2-307a-4630-b27d-cc8eecf99c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:06.343113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc0bf2-307a-4630-b27d-cc8eecf99c28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:06.343397 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc0bf2-307a-4630-b27d-cc8eecf99c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:06.343674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc0bf2-307a-4630-b27d-cc8eecf99c28 admin admin] PATH_INFO: `/v3/projects/62987c0e52cb45bd96e8a11d9b7346b8/users/303be1fec37a46b9a84c425930107bf2/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:06.359974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc0bf2-307a-4630-b27d-cc8eecf99c28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62987c0e52cb45bd96e8a11d9b7346b8, user_id=303be1fec37a46b9a84c425930107bf2, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:06.362854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc0bf2-307a-4630-b27d-cc8eecf99c28 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:06.365748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:06.378810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1011/2038] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:06 2022] PUT /identity/v3/projects/62987c0e52cb45bd96e8a11d9b7346b8/users/303be1fec37a46b9a84c425930107bf2/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:06.382759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf602e-cbb5-4741-9bdd-cdc2a044a481 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:06.393734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf602e-cbb5-4741-9bdd-cdc2a044a481 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:06.450438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f277a5-ee01-4e8d-a3ec-90a8a0f06705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:06.451642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f277a5-ee01-4e8d-a3ec-90a8a0f06705 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:06.451940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f277a5-ee01-4e8d-a3ec-90a8a0f06705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:06.452210 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f277a5-ee01-4e8d-a3ec-90a8a0f06705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:06.452894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f277a5-ee01-4e8d-a3ec-90a8a0f06705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:06.454472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f277a5-ee01-4e8d-a3ec-90a8a0f06705 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:06.464947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1028/2039] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:06.469260 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc0bf2-307a-4630-b27d-cc8eecf99c28 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:06.479316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc0bf2-307a-4630-b27d-cc8eecf99c28 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:06.484400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e11699-f679-405b-bf66-2fa042eaca8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:06.485055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e11699-f679-405b-bf66-2fa042eaca8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:06.485330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e11699-f679-405b-bf66-2fa042eaca8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:06.485598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e11699-f679-405b-bf66-2fa042eaca8f admin admin] PATH_INFO: `/v3/projects/62987c0e52cb45bd96e8a11d9b7346b8/users/303be1fec37a46b9a84c425930107bf2/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:06.487460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e11699-f679-405b-bf66-2fa042eaca8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=62987c0e52cb45bd96e8a11d9b7346b8, user_id=303be1fec37a46b9a84c425930107bf2, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:06.490137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e11699-f679-405b-bf66-2fa042eaca8f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:06.500767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1012/2040] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:06 2022] PUT /identity/v3/projects/62987c0e52cb45bd96e8a11d9b7346b8/users/303be1fec37a46b9a84c425930107bf2/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:06.511712 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b220563-6eb7-4b72-9ae3-6bb563bae0af None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:06.512121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b220563-6eb7-4b72-9ae3-6bb563bae0af None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:06.512404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b220563-6eb7-4b72-9ae3-6bb563bae0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:06.539873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-8b220563-6eb7-4b72-9ae3-6bb563bae0af None None] MFA Rules not processed for user `303be1fec37a46b9a84c425930107bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:06.541092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8b220563-6eb7-4b72-9ae3-6bb563bae0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:06.614394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1029/2041] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:06.625983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e11699-f679-405b-bf66-2fa042eaca8f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:06.627707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c9e11699-f679-405b-bf66-2fa042eaca8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:06.689475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e11699-f679-405b-bf66-2fa042eaca8f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:06.693768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0178a-b600-463a-929c-856bf2c7f131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:06.694384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0178a-b600-463a-929c-856bf2c7f131 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:06.694665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0178a-b600-463a-929c-856bf2c7f131 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:06.694943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0178a-b600-463a-929c-856bf2c7f131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:06.704615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0178a-b600-463a-929c-856bf2c7f131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:06.706071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0178a-b600-463a-929c-856bf2c7f131 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:06.716079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1013/2042] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:07.307462 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b220563-6eb7-4b72-9ae3-6bb563bae0af None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:07.317255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b220563-6eb7-4b72-9ae3-6bb563bae0af None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:07.320919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e9b2f-dcb6-4608-b4c4-90e75d10e649 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:07.321516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e9b2f-dcb6-4608-b4c4-90e75d10e649 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:07.321798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e9b2f-dcb6-4608-b4c4-90e75d10e649 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:07.322072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e9b2f-dcb6-4608-b4c4-90e75d10e649 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:07.347932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:07.380158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e9b2f-dcb6-4608-b4c4-90e75d10e649 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:07.381530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e9b2f-dcb6-4608-b4c4-90e75d10e649 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:07.391062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1030/2043] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:15.603658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0178a-b600-463a-929c-856bf2c7f131 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:15.605180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-efe0178a-b600-463a-929c-856bf2c7f131 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:15.695650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0178a-b600-463a-929c-856bf2c7f131 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:15.701849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:15.702731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:15.703438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:15.703970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] PATH_INFO: `/v3/users/51108c968bba453da457f136a34f6527` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:15.705499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=51108c968bba453da457f136a34f6527)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:15.708114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:15.770152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51108c968bba453da457f136a34f6527'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:15.770152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51108c968bba453da457f136a34f6527'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:15.778581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51108c968bba453da457f136a34f6527'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:15.779238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51108c968bba453da457f136a34f6527'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:15.791351 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51108c968bba453da457f136a34f6527'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:15.814667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1014/2044] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:15 2022] DELETE /identity/v3/users/51108c968bba453da457f136a34f6527 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:15.818977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648c2e4-609b-4f63-ad9c-bc0efffc72fa None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:15.819474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648c2e4-609b-4f63-ad9c-bc0efffc72fa None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:15.819555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648c2e4-609b-4f63-ad9c-bc0efffc72fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:15.842523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7648c2e4-609b-4f63-ad9c-bc0efffc72fa None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:15.843152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7648c2e4-609b-4f63-ad9c-bc0efffc72fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:15.915746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:15.917474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:15.926044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:15.941817 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1031/2045] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:15.950890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648c2e4-609b-4f63-ad9c-bc0efffc72fa None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:15.953086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7648c2e4-609b-4f63-ad9c-bc0efffc72fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:16.013635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f2aef-f519-4586-b7a7-3af248447c2e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:16.021175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:16.022332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:16.022823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:16.023283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] PATH_INFO: `/v3/users/5cbc407e89d044698fb4c12112234542` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:16.024708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cbc407e89d044698fb4c12112234542)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:16.028095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:16.032423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648c2e4-609b-4f63-ad9c-bc0efffc72fa None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:16.037767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de530e-3045-4077-b3af-cffd367a7033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:16.039286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de530e-3045-4077-b3af-cffd367a7033 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:16.039286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de530e-3045-4077-b3af-cffd367a7033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:16.039286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de530e-3045-4077-b3af-cffd367a7033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:16.052874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de530e-3045-4077-b3af-cffd367a7033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:16.055383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de530e-3045-4077-b3af-cffd367a7033 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:16.072062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1032/2046] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:22:16.108015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cbc407e89d044698fb4c12112234542'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:16.108877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cbc407e89d044698fb4c12112234542'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:16.123609 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cbc407e89d044698fb4c12112234542'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:16.124717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cbc407e89d044698fb4c12112234542'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:16.137551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cbc407e89d044698fb4c12112234542'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:16.165835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1015/2047] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:15 2022] DELETE /identity/v3/users/5cbc407e89d044698fb4c12112234542 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:16.170705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b0a5d-52c6-448e-903e-040fa5a44af7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:16.170989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b0a5d-52c6-448e-903e-040fa5a44af7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:16.171257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b0a5d-52c6-448e-903e-040fa5a44af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:16.195145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ff4b0a5d-52c6-448e-903e-040fa5a44af7 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:16.196430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ff4b0a5d-52c6-448e-903e-040fa5a44af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:16.305732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1033/2048] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:16.337205 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:16.340056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:16.434148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7629f-9657-47b1-9da4-644be5b0b1f2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:16.441045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdd3cd-7ee5-40b5-b8ba-0b168fdb4a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:16.441804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdd3cd-7ee5-40b5-b8ba-0b168fdb4a18 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:16.442182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdd3cd-7ee5-40b5-b8ba-0b168fdb4a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:16.442588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdd3cd-7ee5-40b5-b8ba-0b168fdb4a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:16.460793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdd3cd-7ee5-40b5-b8ba-0b168fdb4a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:16.462366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdd3cd-7ee5-40b5-b8ba-0b168fdb4a18 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:16.474883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1016/2049] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:16.927242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b0a5d-52c6-448e-903e-040fa5a44af7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:16.928818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ff4b0a5d-52c6-448e-903e-040fa5a44af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:16.946613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b0a5d-52c6-448e-903e-040fa5a44af7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:16.957140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:16.958139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:16.958683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:16.959172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] PATH_INFO: `/v3/projects/fee0e87f87b44bdba46c0b23f3e58ce6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:16.960534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee0e87f87b44bdba46c0b23f3e58ce6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:16.962630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:16.970733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:17.062835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee0e87f87b44bdba46c0b23f3e58ce6', 'request_id': 'req-b47cb742-9944-463b-af25-6a7255362fb0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.073856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee0e87f87b44bdba46c0b23f3e58ce6', 'request_id': 'req-b47cb742-9944-463b-af25-6a7255362fb0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.085781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee0e87f87b44bdba46c0b23f3e58ce6', 'request_id': 'req-b47cb742-9944-463b-af25-6a7255362fb0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.108410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] The token cache is being invalidate because project fee0e87f87b44bdba46c0b23f3e58ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:17.109036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.110625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.114855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1034/2050] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:16 2022] DELETE /identity/v3/projects/fee0e87f87b44bdba46c0b23f3e58ce6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:17.118574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdd3cd-7ee5-40b5-b8ba-0b168fdb4a18 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:17.147784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdd3cd-7ee5-40b5-b8ba-0b168fdb4a18 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:17.171705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:17.173909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:17.186093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:17.270853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:17.273023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:17.274570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:17.275041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] PATH_INFO: `/v3/users/24fb492aa29845e6add5951b4214a0b8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:17.276903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24fb492aa29845e6add5951b4214a0b8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:17.279929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:17.306588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47cb742-9944-463b-af25-6a7255362fb0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:17.315203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:17.315890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:17.316177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:17.316448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] PATH_INFO: `/v3/projects/5e68b78e3ae245a0bf612436f39aa76b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:17.318355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e68b78e3ae245a0bf612436f39aa76b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:17.320367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:17.382662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24fb492aa29845e6add5951b4214a0b8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.383420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24fb492aa29845e6add5951b4214a0b8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.397169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24fb492aa29845e6add5951b4214a0b8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.398640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24fb492aa29845e6add5951b4214a0b8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.413286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24fb492aa29845e6add5951b4214a0b8'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.444127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e68b78e3ae245a0bf612436f39aa76b', 'request_id': 'req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.447480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1017/2051] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:17 2022] DELETE /identity/v3/users/24fb492aa29845e6add5951b4214a0b8 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:17.460402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e68b78e3ae245a0bf612436f39aa76b', 'request_id': 'req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.477500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e68b78e3ae245a0bf612436f39aa76b', 'request_id': 'req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.496322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] The token cache is being invalidate because project 5e68b78e3ae245a0bf612436f39aa76b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:17.496654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.497915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.500494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1035/2052] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:17 2022] DELETE /identity/v3/projects/5e68b78e3ae245a0bf612436f39aa76b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:17.504578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:17.506258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:17.641573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22e90c-25e3-4835-b540-bc846fb3b931 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:17.647841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:17.648296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:17.648602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:17.648851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] PATH_INFO: `/v3/users/516a72f40ce342e485bd6558c3405311` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:17.650553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=516a72f40ce342e485bd6558c3405311)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:17.660184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:17.774231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516a72f40ce342e485bd6558c3405311'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.774983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516a72f40ce342e485bd6558c3405311'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.777221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:17.789155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516a72f40ce342e485bd6558c3405311'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.789875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516a72f40ce342e485bd6558c3405311'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.800475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d2686ea6-4d5e-4fc9-86bf-b1b76ba2b621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516a72f40ce342e485bd6558c3405311'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:17.828190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1018/2053] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:17 2022] DELETE /identity/v3/users/516a72f40ce342e485bd6558c3405311 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:17.930762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:17.932246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:17.990021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:18.058693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7d724-81dc-4ebb-94e7-6ae31dd7a102 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:18.141242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:18.144201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:18.144611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:18.144986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] PATH_INFO: `/v3/projects/9d075708835c4e2c9df8905d66b779af` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:18.146331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d075708835c4e2c9df8905d66b779af)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:18.148391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:18.212662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d075708835c4e2c9df8905d66b779af', 'request_id': 'req-497cb641-dab5-40ca-b81b-2bce3e4c63bc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.224451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d075708835c4e2c9df8905d66b779af', 'request_id': 'req-497cb641-dab5-40ca-b81b-2bce3e4c63bc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.236926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d075708835c4e2c9df8905d66b779af', 'request_id': 'req-497cb641-dab5-40ca-b81b-2bce3e4c63bc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.264873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] The token cache is being invalidate because project 9d075708835c4e2c9df8905d66b779af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:18.265171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.272678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.274685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1036/2054] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:17 2022] DELETE /identity/v3/projects/9d075708835c4e2c9df8905d66b779af => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:18.280232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24ce75-7ab0-42d8-b5a5-3bb9d0086220 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:18.280716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24ce75-7ab0-42d8-b5a5-3bb9d0086220 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:18.280804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24ce75-7ab0-42d8-b5a5-3bb9d0086220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:18.344946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:18.347179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:18.350706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2e24ce75-7ab0-42d8-b5a5-3bb9d0086220 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:18.361748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2e24ce75-7ab0-42d8-b5a5-3bb9d0086220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:18.529871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497cb641-dab5-40ca-b81b-2bce3e4c63bc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:18.542680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1019/2055] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:18.550628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f67a2b-4576-40af-9b2a-40dd47acff8e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:18.550982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:18.551463 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f67a2b-4576-40af-9b2a-40dd47acff8e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:18.551672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:18.551947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:18.552354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] PATH_INFO: `/v3/projects/de2ae749b1004066a7f581d639191faf` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:18.552655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f67a2b-4576-40af-9b2a-40dd47acff8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:18.553587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2ae749b1004066a7f581d639191faf)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:18.556078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:18.562170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:18.606187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-99f67a2b-4576-40af-9b2a-40dd47acff8e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:18.613493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-99f67a2b-4576-40af-9b2a-40dd47acff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:18.647473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2ae749b1004066a7f581d639191faf', 'request_id': 'req-3d38fd75-417e-4fb4-8596-11a957fa257e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.662193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2ae749b1004066a7f581d639191faf', 'request_id': 'req-3d38fd75-417e-4fb4-8596-11a957fa257e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.681755 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2ae749b1004066a7f581d639191faf', 'request_id': 'req-3d38fd75-417e-4fb4-8596-11a957fa257e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.699579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] The token cache is being invalidate because project de2ae749b1004066a7f581d639191faf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:18.700825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.733452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3d38fd75-417e-4fb4-8596-11a957fa257e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:18.739119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1037/2056] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:18 2022] DELETE /identity/v3/projects/de2ae749b1004066a7f581d639191faf => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:18.747750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1311e3f-8652-454d-97d4-2f0ada69ad2c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:18.764637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1311e3f-8652-454d-97d4-2f0ada69ad2c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:18.765205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1311e3f-8652-454d-97d4-2f0ada69ad2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:18.803794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1020/2057] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:18.808790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f67a2b-4576-40af-9b2a-40dd47acff8e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:18.834541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f67a2b-4576-40af-9b2a-40dd47acff8e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:18.834804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b1311e3f-8652-454d-97d4-2f0ada69ad2c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:18.836130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b1311e3f-8652-454d-97d4-2f0ada69ad2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:18.844743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea9702-e257-450b-a563-5a225b5a73b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:18.846137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea9702-e257-450b-a563-5a225b5a73b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:18.846137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea9702-e257-450b-a563-5a225b5a73b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:18.846378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea9702-e257-450b-a563-5a225b5a73b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:18.847613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea9702-e257-450b-a563-5a225b5a73b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:18.848218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea9702-e257-450b-a563-5a225b5a73b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:18.852792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea9702-e257-450b-a563-5a225b5a73b3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:18.864165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1038/2058] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:18.866271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce363cda-8b7d-4066-beff-2e8d6cb15890 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:18.866555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce363cda-8b7d-4066-beff-2e8d6cb15890 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:18.866832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce363cda-8b7d-4066-beff-2e8d6cb15890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:18.897481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1021/2059] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:22:18.899555 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea9702-e257-450b-a563-5a225b5a73b3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:18.905095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ce363cda-8b7d-4066-beff-2e8d6cb15890 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:18.906315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ce363cda-8b7d-4066-beff-2e8d6cb15890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:18.917688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1039/2060] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:18.918823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce363cda-8b7d-4066-beff-2e8d6cb15890 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:18.925495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea9702-e257-450b-a563-5a225b5a73b3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:18.939903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597293dd-da82-4aaf-aa77-e574277612c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:18.940637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597293dd-da82-4aaf-aa77-e574277612c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:18.941041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597293dd-da82-4aaf-aa77-e574277612c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:18.941547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597293dd-da82-4aaf-aa77-e574277612c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:18.942415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597293dd-da82-4aaf-aa77-e574277612c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:18.942742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce363cda-8b7d-4066-beff-2e8d6cb15890 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:18.949302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597293dd-da82-4aaf-aa77-e574277612c2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:18.955719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d055418-6ec7-489b-b971-df208cd281fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:18.956544 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d055418-6ec7-489b-b971-df208cd281fd admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:18.957080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d055418-6ec7-489b-b971-df208cd281fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:18.957354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d055418-6ec7-489b-b971-df208cd281fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:18.958415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d055418-6ec7-489b-b971-df208cd281fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:18.958415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d055418-6ec7-489b-b971-df208cd281fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:18.964185 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d055418-6ec7-489b-b971-df208cd281fd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.005099 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1040/2061] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:22:19.006852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d055418-6ec7-489b-b971-df208cd281fd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.025176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1022/2062] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:18 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:19.029616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597293dd-da82-4aaf-aa77-e574277612c2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.036879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d055418-6ec7-489b-b971-df208cd281fd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.121447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:19.189016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597293dd-da82-4aaf-aa77-e574277612c2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.193578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67224a53-b89e-4047-913d-2560229cac51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.194901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28217d41-1d62-4459-be11-abfa1c1eb586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.195857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67224a53-b89e-4047-913d-2560229cac51 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.196208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28217d41-1d62-4459-be11-abfa1c1eb586 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.197970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28217d41-1d62-4459-be11-abfa1c1eb586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.197970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28217d41-1d62-4459-be11-abfa1c1eb586 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.198531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28217d41-1d62-4459-be11-abfa1c1eb586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.201579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28217d41-1d62-4459-be11-abfa1c1eb586 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.224365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67224a53-b89e-4047-913d-2560229cac51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.224661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67224a53-b89e-4047-913d-2560229cac51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.225476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67224a53-b89e-4047-913d-2560229cac51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.227458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67224a53-b89e-4047-913d-2560229cac51 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.240513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1023/2063] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:19 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:19.245897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28217d41-1d62-4459-be11-abfa1c1eb586 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.260030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28217d41-1d62-4459-be11-abfa1c1eb586 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.272591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197716c2-f7cc-4baa-ba54-b0d36b55673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.273530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197716c2-f7cc-4baa-ba54-b0d36b55673e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.274485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197716c2-f7cc-4baa-ba54-b0d36b55673e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.274941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197716c2-f7cc-4baa-ba54-b0d36b55673e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.276783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197716c2-f7cc-4baa-ba54-b0d36b55673e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.280160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197716c2-f7cc-4baa-ba54-b0d36b55673e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.298261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1041/2064] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:19 2022] POST /identity/v3/projects => generated 373 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:22:19.301515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1024/2065] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:19.303032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67224a53-b89e-4047-913d-2560229cac51 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.306169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197716c2-f7cc-4baa-ba54-b0d36b55673e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.324804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197716c2-f7cc-4baa-ba54-b0d36b55673e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.428307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:19.463532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67224a53-b89e-4047-913d-2560229cac51 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.471212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52df66c-2543-4d8c-b3d5-cdb6a56cf701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.472116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52df66c-2543-4d8c-b3d5-cdb6a56cf701 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.472457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52df66c-2543-4d8c-b3d5-cdb6a56cf701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.473082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52df66c-2543-4d8c-b3d5-cdb6a56cf701 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.473551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52df66c-2543-4d8c-b3d5-cdb6a56cf701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.476273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52df66c-2543-4d8c-b3d5-cdb6a56cf701 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.483874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d1ebd-7ec0-4d03-9d3a-0b8ce4aa77bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.486635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d1ebd-7ec0-4d03-9d3a-0b8ce4aa77bd admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.486924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d1ebd-7ec0-4d03-9d3a-0b8ce4aa77bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.487203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d1ebd-7ec0-4d03-9d3a-0b8ce4aa77bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.490306 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d1ebd-7ec0-4d03-9d3a-0b8ce4aa77bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.498118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d1ebd-7ec0-4d03-9d3a-0b8ce4aa77bd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.509852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1042/2066] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:19 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:19.512132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52df66c-2543-4d8c-b3d5-cdb6a56cf701 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.516011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f52df66c-2543-4d8c-b3d5-cdb6a56cf701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:19.523650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1025/2067] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:19.524623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d1ebd-7ec0-4d03-9d3a-0b8ce4aa77bd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.545247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d1ebd-7ec0-4d03-9d3a-0b8ce4aa77bd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.569092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15507fdc-42fe-4b92-8159-9e6881c9e9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.569690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15507fdc-42fe-4b92-8159-9e6881c9e9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.569967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15507fdc-42fe-4b92-8159-9e6881c9e9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.570240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15507fdc-42fe-4b92-8159-9e6881c9e9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.570889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15507fdc-42fe-4b92-8159-9e6881c9e9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.585193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15507fdc-42fe-4b92-8159-9e6881c9e9cb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.596515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52df66c-2543-4d8c-b3d5-cdb6a56cf701 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.600410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd89563-a36a-4e60-b6a6-a3e2edc6bb30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.601032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd89563-a36a-4e60-b6a6-a3e2edc6bb30 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.601312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd89563-a36a-4e60-b6a6-a3e2edc6bb30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.601584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd89563-a36a-4e60-b6a6-a3e2edc6bb30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.602280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1026/2068] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:19.603507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dcd89563-a36a-4e60-b6a6-a3e2edc6bb30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:19.603925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15507fdc-42fe-4b92-8159-9e6881c9e9cb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.618017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15507fdc-42fe-4b92-8159-9e6881c9e9cb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.623976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effda6c-36a4-4211-8a51-363a73f552f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.625078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effda6c-36a4-4211-8a51-363a73f552f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.625591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effda6c-36a4-4211-8a51-363a73f552f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.626069 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9effda6c-36a4-4211-8a51-363a73f552f4 admin admin] PATH_INFO: `/v3/projects/ce85bd4a365140bcaca7d69ba0e9e1c9/users/09210c478f9e4077bafa00f72678561e/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.645180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effda6c-36a4-4211-8a51-363a73f552f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce85bd4a365140bcaca7d69ba0e9e1c9, user_id=09210c478f9e4077bafa00f72678561e, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.648759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9effda6c-36a4-4211-8a51-363a73f552f4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.671023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1027/2069] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:19 2022] PUT /identity/v3/projects/ce85bd4a365140bcaca7d69ba0e9e1c9/users/09210c478f9e4077bafa00f72678561e/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:19.672387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effda6c-36a4-4211-8a51-363a73f552f4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.681149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd89563-a36a-4e60-b6a6-a3e2edc6bb30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.682630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd89563-a36a-4e60-b6a6-a3e2edc6bb30 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.693547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1043/2070] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:19.695324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c620c4-7d11-4b7b-98a3-2ac0811327af None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.695622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c620c4-7d11-4b7b-98a3-2ac0811327af None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.695910 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c620c4-7d11-4b7b-98a3-2ac0811327af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.760812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-56c620c4-7d11-4b7b-98a3-2ac0811327af None None] MFA Rules not processed for user `09210c478f9e4077bafa00f72678561e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:19.762065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-56c620c4-7d11-4b7b-98a3-2ac0811327af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:19.795480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9effda6c-36a4-4211-8a51-363a73f552f4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.801827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bd47d-d095-44ca-af99-f3c9c6fcde3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.803047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bd47d-d095-44ca-af99-f3c9c6fcde3e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.803441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bd47d-d095-44ca-af99-f3c9c6fcde3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.803811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bd47d-d095-44ca-af99-f3c9c6fcde3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.804618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bd47d-d095-44ca-af99-f3c9c6fcde3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.806657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bd47d-d095-44ca-af99-f3c9c6fcde3e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.819104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1028/2071] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:19.823219 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bd47d-d095-44ca-af99-f3c9c6fcde3e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.835958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bd47d-d095-44ca-af99-f3c9c6fcde3e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.843821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cec85-3656-484d-8a4b-539830ccc028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:19.844744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cec85-3656-484d-8a4b-539830ccc028 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.845127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cec85-3656-484d-8a4b-539830ccc028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.845591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cec85-3656-484d-8a4b-539830ccc028 admin admin] PATH_INFO: `/v3/projects/c46922ef081642f6a3d8ad5d02e4155a/users/1f8e995175ec4c6583e1ad0c7fb902ab/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.848706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:19.874633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cec85-3656-484d-8a4b-539830ccc028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46922ef081642f6a3d8ad5d02e4155a, user_id=1f8e995175ec4c6583e1ad0c7fb902ab, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:19.875687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1044/2072] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:19.876471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cec85-3656-484d-8a4b-539830ccc028 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:19.877584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c620c4-7d11-4b7b-98a3-2ac0811327af None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:19.892369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c620c4-7d11-4b7b-98a3-2ac0811327af None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:19.898648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1029/2073] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:19 2022] PUT /identity/v3/projects/c46922ef081642f6a3d8ad5d02e4155a/users/1f8e995175ec4c6583e1ad0c7fb902ab/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:19.905976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbba05-2bd2-41c0-a594-1e7d3131fa1b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:19.906347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbba05-2bd2-41c0-a594-1e7d3131fa1b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:19.906745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbba05-2bd2-41c0-a594-1e7d3131fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:19.921815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:19.945146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5cbbba05-2bd2-41c0-a594-1e7d3131fa1b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:19.947237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5cbbba05-2bd2-41c0-a594-1e7d3131fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:20.007895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d667feb-94ab-4cd3-a056-ce0490463b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:20.008675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d667feb-94ab-4cd3-a056-ce0490463b3e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:20.008880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d667feb-94ab-4cd3-a056-ce0490463b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:20.009158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d667feb-94ab-4cd3-a056-ce0490463b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:20.011736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d667feb-94ab-4cd3-a056-ce0490463b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:20.017470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d667feb-94ab-4cd3-a056-ce0490463b3e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:20.041351 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1030/2074] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:20.053063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbba05-2bd2-41c0-a594-1e7d3131fa1b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:20.056209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5cbbba05-2bd2-41c0-a594-1e7d3131fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:20.077925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1045/2075] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:19 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:20.085021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d667feb-94ab-4cd3-a056-ce0490463b3e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:20.195498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d667feb-94ab-4cd3-a056-ce0490463b3e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:20.202171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddb0d2-d109-4730-88dd-853bf64f5495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:20.202851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddb0d2-d109-4730-88dd-853bf64f5495 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:20.203137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddb0d2-d109-4730-88dd-853bf64f5495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:20.203412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deddb0d2-d109-4730-88dd-853bf64f5495 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:20.204087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddb0d2-d109-4730-88dd-853bf64f5495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:20.204490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbba05-2bd2-41c0-a594-1e7d3131fa1b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:20.209707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deddb0d2-d109-4730-88dd-853bf64f5495 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:20.247471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1046/2076] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:20 2022] POST /identity/v3/users => generated 300 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:22:20.248634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddb0d2-d109-4730-88dd-853bf64f5495 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:20.265145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deddb0d2-d109-4730-88dd-853bf64f5495 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:20.274092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3be1fb-ce46-435b-beb8-47d0686ad7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:20.274879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3be1fb-ce46-435b-beb8-47d0686ad7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:20.275250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3be1fb-ce46-435b-beb8-47d0686ad7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:20.275596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3be1fb-ce46-435b-beb8-47d0686ad7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:20.276360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3be1fb-ce46-435b-beb8-47d0686ad7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:20.278161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3be1fb-ce46-435b-beb8-47d0686ad7d6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:20.290914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1047/2077] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:20.291650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a6cce-f8d5-4fb9-8b19-4fccc299279b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:20.292377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a6cce-f8d5-4fb9-8b19-4fccc299279b service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:20.292921 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a6cce-f8d5-4fb9-8b19-4fccc299279b service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:20.294800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3be1fb-ce46-435b-beb8-47d0686ad7d6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:20.295904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a6cce-f8d5-4fb9-8b19-4fccc299279b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:20.310219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3be1fb-ce46-435b-beb8-47d0686ad7d6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:20.311730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a6cce-f8d5-4fb9-8b19-4fccc299279b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:20.313392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a6cce-f8d5-4fb9-8b19-4fccc299279b service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:20.316579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff9585-4541-4e08-ac34-5631dc1d89d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:20.317655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff9585-4541-4e08-ac34-5631dc1d89d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:20.318168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff9585-4541-4e08-ac34-5631dc1d89d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:20.318640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff9585-4541-4e08-ac34-5631dc1d89d1 admin admin] PATH_INFO: `/v3/projects/05b49277c74749a4af8848591aff75bf/users/46cd4f7c54024e3683e4eefb30de1bbe/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:20.333075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1031/2078] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:20.347558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff9585-4541-4e08-ac34-5631dc1d89d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b49277c74749a4af8848591aff75bf, user_id=46cd4f7c54024e3683e4eefb30de1bbe, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:20.350272 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff9585-4541-4e08-ac34-5631dc1d89d1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:20.371078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1048/2079] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:20 2022] PUT /identity/v3/projects/05b49277c74749a4af8848591aff75bf/users/46cd4f7c54024e3683e4eefb30de1bbe/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:20.376634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a6cce-f8d5-4fb9-8b19-4fccc299279b service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:20.389801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a6cce-f8d5-4fb9-8b19-4fccc299279b service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:20.474528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:20.512287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37d898-7011-425b-8ebe-df2b56a7ef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:20.514723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37d898-7011-425b-8ebe-df2b56a7ef14 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:20.517070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37d898-7011-425b-8ebe-df2b56a7ef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:20.517641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37d898-7011-425b-8ebe-df2b56a7ef14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:20.519324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37d898-7011-425b-8ebe-df2b56a7ef14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:20.521765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37d898-7011-425b-8ebe-df2b56a7ef14 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:20.555252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff9585-4541-4e08-ac34-5631dc1d89d1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:20.557095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1032/2080] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:20.570772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff9585-4541-4e08-ac34-5631dc1d89d1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:20.584126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40380e64-7879-4597-a97f-c17f837bd05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:20.584924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380e64-7879-4597-a97f-c17f837bd05d admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:20.585277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380e64-7879-4597-a97f-c17f837bd05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:20.585624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40380e64-7879-4597-a97f-c17f837bd05d admin admin] PATH_INFO: `/v3/projects/05b49277c74749a4af8848591aff75bf/users/46cd4f7c54024e3683e4eefb30de1bbe/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:20.587863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40380e64-7879-4597-a97f-c17f837bd05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b49277c74749a4af8848591aff75bf, user_id=46cd4f7c54024e3683e4eefb30de1bbe, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:20.590879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40380e64-7879-4597-a97f-c17f837bd05d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:20.615064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1049/2081] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:20 2022] PUT /identity/v3/projects/05b49277c74749a4af8848591aff75bf/users/46cd4f7c54024e3683e4eefb30de1bbe/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:20.629368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6515-d8b2-41da-965a-c224a1267f5e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:20.629715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6515-d8b2-41da-965a-c224a1267f5e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:20.630058 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c6515-d8b2-41da-965a-c224a1267f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:20.716120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c74c6515-d8b2-41da-965a-c224a1267f5e None None] MFA Rules not processed for user `46cd4f7c54024e3683e4eefb30de1bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:20.717689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c74c6515-d8b2-41da-965a-c224a1267f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:20.835278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1033/2082] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:20.870205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40380e64-7879-4597-a97f-c17f837bd05d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:20.872495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-40380e64-7879-4597-a97f-c17f837bd05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:20.947507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:20.953273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40380e64-7879-4597-a97f-c17f837bd05d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:20.957955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4476-2e61-4876-ab1c-7563f812d9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:20.958580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4476-2e61-4876-ab1c-7563f812d9ed service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:20.958969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4476-2e61-4876-ab1c-7563f812d9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:20.959402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4476-2e61-4876-ab1c-7563f812d9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:20.970838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4476-2e61-4876-ab1c-7563f812d9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:20.972482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4476-2e61-4876-ab1c-7563f812d9ed service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:21.012783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1050/2083] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:21.110276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c6515-d8b2-41da-965a-c224a1267f5e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:21.192253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c6515-d8b2-41da-965a-c224a1267f5e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:21.204127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:21.204928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:21.205218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:21.205490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] PATH_INFO: `/v3/users/09210c478f9e4077bafa00f72678561e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:21.207840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09210c478f9e4077bafa00f72678561e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:21.209967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:21.274073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09210c478f9e4077bafa00f72678561e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:21.275174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09210c478f9e4077bafa00f72678561e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:21.288659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09210c478f9e4077bafa00f72678561e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:21.290660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09210c478f9e4077bafa00f72678561e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:21.304220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a759098-45f1-487d-a167-92ca57c69a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09210c478f9e4077bafa00f72678561e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:21.306854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:21.328087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1034/2084] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:21 2022] DELETE /identity/v3/users/09210c478f9e4077bafa00f72678561e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:21.333833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92f750-3069-40dd-81b7-8745dbf5e9a4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:21.334119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92f750-3069-40dd-81b7-8745dbf5e9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:21.334395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92f750-3069-40dd-81b7-8745dbf5e9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:21.358913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1b92f750-3069-40dd-81b7-8745dbf5e9a4 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:21.361066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1b92f750-3069-40dd-81b7-8745dbf5e9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:21.420416 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270736b-624d-4d61-a853-b8e9fa511de3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:21.420606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270736b-624d-4d61-a853-b8e9fa511de3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:21.421440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270736b-624d-4d61-a853-b8e9fa511de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:21.471502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1051/2085] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:21.474752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f270736b-624d-4d61-a853-b8e9fa511de3 None None] MFA Rules not processed for user `1f8e995175ec4c6583e1ad0c7fb902ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:21.477481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f270736b-624d-4d61-a853-b8e9fa511de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:21.481982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92f750-3069-40dd-81b7-8745dbf5e9a4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:21.484416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1b92f750-3069-40dd-81b7-8745dbf5e9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:21.585674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92f750-3069-40dd-81b7-8745dbf5e9a4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:21.590422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc369fe-cd16-44f8-9746-54567c915230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:21.591857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc369fe-cd16-44f8-9746-54567c915230 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:21.592240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc369fe-cd16-44f8-9746-54567c915230 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:21.592772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc369fe-cd16-44f8-9746-54567c915230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:21.604401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc369fe-cd16-44f8-9746-54567c915230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:21.605917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc369fe-cd16-44f8-9746-54567c915230 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:21.623124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1052/2086] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:21.630430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1035/2087] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:21.639004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc369fe-cd16-44f8-9746-54567c915230 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:21.642162 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ecc369fe-cd16-44f8-9746-54567c915230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:21.745403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc369fe-cd16-44f8-9746-54567c915230 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:21.763640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e97104-3658-40cc-8dc7-eb44a3f14b67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:21.764156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97104-3658-40cc-8dc7-eb44a3f14b67 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:21.764435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97104-3658-40cc-8dc7-eb44a3f14b67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:21.764705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97104-3658-40cc-8dc7-eb44a3f14b67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:21.791802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e97104-3658-40cc-8dc7-eb44a3f14b67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:21.793355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e97104-3658-40cc-8dc7-eb44a3f14b67 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:21.814478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1053/2088] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:22.127206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270736b-624d-4d61-a853-b8e9fa511de3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:22.128777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f270736b-624d-4d61-a853-b8e9fa511de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:22.182602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270736b-624d-4d61-a853-b8e9fa511de3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:22.182602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:22.182602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:22.182602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:22.183465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] PATH_INFO: `/v3/projects/ce85bd4a365140bcaca7d69ba0e9e1c9` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:22.183465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce85bd4a365140bcaca7d69ba0e9e1c9)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:22.183465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:22.285772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce85bd4a365140bcaca7d69ba0e9e1c9', 'request_id': 'req-1858372e-9d2b-42da-957a-8173ce12847d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.304319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce85bd4a365140bcaca7d69ba0e9e1c9', 'request_id': 'req-1858372e-9d2b-42da-957a-8173ce12847d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.332448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce85bd4a365140bcaca7d69ba0e9e1c9', 'request_id': 'req-1858372e-9d2b-42da-957a-8173ce12847d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.345244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] The token cache is being invalidate because project ce85bd4a365140bcaca7d69ba0e9e1c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:22.346684 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.348107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.355604 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1036/2089] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:22 2022] DELETE /identity/v3/projects/ce85bd4a365140bcaca7d69ba0e9e1c9 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:22.357139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e97104-3658-40cc-8dc7-eb44a3f14b67 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:22.377258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e97104-3658-40cc-8dc7-eb44a3f14b67 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:22.469349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:22.520527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:22.521214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:22.538838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:22.539125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] PATH_INFO: `/v3/users/46cd4f7c54024e3683e4eefb30de1bbe` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:22.540721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46cd4f7c54024e3683e4eefb30de1bbe)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:22.542764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:22.645798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46cd4f7c54024e3683e4eefb30de1bbe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.655640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46cd4f7c54024e3683e4eefb30de1bbe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.657052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46cd4f7c54024e3683e4eefb30de1bbe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.657544 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46cd4f7c54024e3683e4eefb30de1bbe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.666729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46cd4f7c54024e3683e4eefb30de1bbe'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:22.706365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1054/2090] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:22 2022] DELETE /identity/v3/users/46cd4f7c54024e3683e4eefb30de1bbe => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:23.092272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.094355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:23.115047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858372e-9d2b-42da-957a-8173ce12847d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:23.153057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.155040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:23.169084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:23.177015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:23.177766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.178347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.178825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] PATH_INFO: `/v3/projects/05b49277c74749a4af8848591aff75bf` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.180486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b49277c74749a4af8848591aff75bf)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:23.183623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:23.231500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554a92d-0132-49be-9672-36ab5890e6a1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:23.242797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b49277c74749a4af8848591aff75bf', 'request_id': 'req-08950964-31ec-42cb-bd4d-d68aaad7b843'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.249989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b49277c74749a4af8848591aff75bf', 'request_id': 'req-08950964-31ec-42cb-bd4d-d68aaad7b843'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.261571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b49277c74749a4af8848591aff75bf', 'request_id': 'req-08950964-31ec-42cb-bd4d-d68aaad7b843'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.273525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] The token cache is being invalidate because project 05b49277c74749a4af8848591aff75bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:23.274047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.274703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-08950964-31ec-42cb-bd4d-d68aaad7b843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.276284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1037/2091] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:23 2022] DELETE /identity/v3/projects/05b49277c74749a4af8848591aff75bf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:23.281630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298017ac-380d-4215-b9a7-349052fd5ad3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.281916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298017ac-380d-4215-b9a7-349052fd5ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.282190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298017ac-380d-4215-b9a7-349052fd5ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.297506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:23.298144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.298422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.298695 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] PATH_INFO: `/v3/users/6d965e6045594412b6614ab697ead08c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.300157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d965e6045594412b6614ab697ead08c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:23.301993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-298017ac-380d-4215-b9a7-349052fd5ad3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:23.302141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:23.303210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-298017ac-380d-4215-b9a7-349052fd5ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:23.306967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:23.311653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1038/2092] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:23.319410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df8eaf-1b5c-47ce-831c-bc3061bba2ad None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.319698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df8eaf-1b5c-47ce-831c-bc3061bba2ad None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.320002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df8eaf-1b5c-47ce-831c-bc3061bba2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.353028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c2df8eaf-1b5c-47ce-831c-bc3061bba2ad None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:23.353028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c2df8eaf-1b5c-47ce-831c-bc3061bba2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:23.354451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1039/2093] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:23.358859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df8eaf-1b5c-47ce-831c-bc3061bba2ad None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.368784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df8eaf-1b5c-47ce-831c-bc3061bba2ad None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:23.374542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:23.375152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.375435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.375710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.376392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:23.376699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:23.377213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d965e6045594412b6614ab697ead08c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.378308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:23.385808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d965e6045594412b6614ab697ead08c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.386700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d965e6045594412b6614ab697ead08c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.387340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d965e6045594412b6614ab697ead08c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.395162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1040/2094] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:22:23.396282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.398036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:23.400359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-15caa6d4-9405-4dac-a97d-8888035edc10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d965e6045594412b6614ab697ead08c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:23.412581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91386a22-32b8-41b9-9b1a-f1356a6efefc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:23.418611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382b410-66f0-4c19-afef-6235553917f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:23.419458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382b410-66f0-4c19-afef-6235553917f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.419795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382b410-66f0-4c19-afef-6235553917f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.420113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382b410-66f0-4c19-afef-6235553917f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.421140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1055/2095] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:23 2022] DELETE /identity/v3/users/6d965e6045594412b6614ab697ead08c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:23.421264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382b410-66f0-4c19-afef-6235553917f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:23.423497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382b410-66f0-4c19-afef-6235553917f4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:23.425513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aaf6b0-96d1-46c7-8bc6-6c2e272e4f36 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.425858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aaf6b0-96d1-46c7-8bc6-6c2e272e4f36 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.426071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aaf6b0-96d1-46c7-8bc6-6c2e272e4f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.453082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c5aaf6b0-96d1-46c7-8bc6-6c2e272e4f36 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:23.454686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5aaf6b0-96d1-46c7-8bc6-6c2e272e4f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:23.464356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1041/2096] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:23 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:22:23.468857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382b410-66f0-4c19-afef-6235553917f4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.527464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1056/2097] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:23.537616 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aaf6b0-96d1-46c7-8bc6-6c2e272e4f36 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.539351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5aaf6b0-96d1-46c7-8bc6-6c2e272e4f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:23.539688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382b410-66f0-4c19-afef-6235553917f4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:23.545890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecc890-f996-4bfe-8292-6d200ec1d3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:23.546504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecc890-f996-4bfe-8292-6d200ec1d3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.546786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecc890-f996-4bfe-8292-6d200ec1d3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.547059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecc890-f996-4bfe-8292-6d200ec1d3f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.547836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ecc890-f996-4bfe-8292-6d200ec1d3f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:23.549727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ecc890-f996-4bfe-8292-6d200ec1d3f6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:23.579251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1042/2098] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:23 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:23.580695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecc890-f996-4bfe-8292-6d200ec1d3f6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.592802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecc890-f996-4bfe-8292-6d200ec1d3f6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:23.599342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ba4ea-5a15-40f7-8f3d-2eb68871103f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:23.600236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ba4ea-5a15-40f7-8f3d-2eb68871103f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.600519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ba4ea-5a15-40f7-8f3d-2eb68871103f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.600902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ba4ea-5a15-40f7-8f3d-2eb68871103f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.601563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ba4ea-5a15-40f7-8f3d-2eb68871103f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:23.603171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ba4ea-5a15-40f7-8f3d-2eb68871103f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:23.608927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aaf6b0-96d1-46c7-8bc6-6c2e272e4f36 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:23.615073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1043/2099] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:23.620092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ba4ea-5a15-40f7-8f3d-2eb68871103f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.633056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ba4ea-5a15-40f7-8f3d-2eb68871103f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:23.640444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b21f1b-9ca5-4054-9776-ce164b383035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:23.640952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b21f1b-9ca5-4054-9776-ce164b383035 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.641234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b21f1b-9ca5-4054-9776-ce164b383035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.641502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b21f1b-9ca5-4054-9776-ce164b383035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.642350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b21f1b-9ca5-4054-9776-ce164b383035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:23.644913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b21f1b-9ca5-4054-9776-ce164b383035 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:23.657365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:23.659801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1044/2100] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:23.671838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416bf1d-0493-4a79-af5e-2e993541b73c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:23.672485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416bf1d-0493-4a79-af5e-2e993541b73c service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.672766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416bf1d-0493-4a79-af5e-2e993541b73c service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.673039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416bf1d-0493-4a79-af5e-2e993541b73c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.676660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b21f1b-9ca5-4054-9776-ce164b383035 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.685234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:23.685986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416bf1d-0493-4a79-af5e-2e993541b73c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:23.687841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416bf1d-0493-4a79-af5e-2e993541b73c service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:23.689261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b21f1b-9ca5-4054-9776-ce164b383035 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:23.697489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fbe9d-28f9-47c3-90b0-0b0ae443ea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:23.698248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fbe9d-28f9-47c3-90b0-0b0ae443ea75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.698855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fbe9d-28f9-47c3-90b0-0b0ae443ea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.699407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fbe9d-28f9-47c3-90b0-0b0ae443ea75 admin admin] PATH_INFO: `/v3/projects/fc30e1b8d238499baee1a5f94778044d/users/07de1fb3841b466ca357d48064b3a3ef/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.716137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1057/2101] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:22:23.720072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393168-a670-49d0-a687-eadb519d8a48 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.720534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393168-a670-49d0-a687-eadb519d8a48 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.720793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393168-a670-49d0-a687-eadb519d8a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.722550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fbe9d-28f9-47c3-90b0-0b0ae443ea75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc30e1b8d238499baee1a5f94778044d, user_id=07de1fb3841b466ca357d48064b3a3ef, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:23.725269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fbe9d-28f9-47c3-90b0-0b0ae443ea75 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:23.744187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b9393168-a670-49d0-a687-eadb519d8a48 None None] MFA Rules not processed for user `1f8e995175ec4c6583e1ad0c7fb902ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:23.745522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b9393168-a670-49d0-a687-eadb519d8a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:23.755098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1045/2102] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:23 2022] PUT /identity/v3/projects/fc30e1b8d238499baee1a5f94778044d/users/07de1fb3841b466ca357d48064b3a3ef/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:23.767902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd06de-c2f4-4a77-864f-0087c17a0299 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:23.767902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd06de-c2f4-4a77-864f-0087c17a0299 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:23.767902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd06de-c2f4-4a77-864f-0087c17a0299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:23.804730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-efcd06de-c2f4-4a77-864f-0087c17a0299 None None] MFA Rules not processed for user `07de1fb3841b466ca357d48064b3a3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:23.806268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-efcd06de-c2f4-4a77-864f-0087c17a0299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:23.824599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1058/2103] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:23.927391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393168-a670-49d0-a687-eadb519d8a48 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:23.929894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1046/2104] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:24.016438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393168-a670-49d0-a687-eadb519d8a48 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:24.021265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75047a2c-1545-468d-a34b-c57d5990ade7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:24.021938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75047a2c-1545-468d-a34b-c57d5990ade7 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.022278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75047a2c-1545-468d-a34b-c57d5990ade7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.022608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75047a2c-1545-468d-a34b-c57d5990ade7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.023570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75047a2c-1545-468d-a34b-c57d5990ade7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:24.025335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75047a2c-1545-468d-a34b-c57d5990ade7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:24.074560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1059/2105] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:23 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:24.074744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd06de-c2f4-4a77-864f-0087c17a0299 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:24.075982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-efcd06de-c2f4-4a77-864f-0087c17a0299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:24.076495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75047a2c-1545-468d-a34b-c57d5990ade7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:24.122349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd06de-c2f4-4a77-864f-0087c17a0299 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:24.167811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75047a2c-1545-468d-a34b-c57d5990ade7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:24.173987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab065b-74b0-4ddd-a403-bbf1de205173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:24.174961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab065b-74b0-4ddd-a403-bbf1de205173 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.175377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab065b-74b0-4ddd-a403-bbf1de205173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.175648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab065b-74b0-4ddd-a403-bbf1de205173 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.180918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab065b-74b0-4ddd-a403-bbf1de205173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:24.180918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab065b-74b0-4ddd-a403-bbf1de205173 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:24.208903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1060/2106] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:24 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:24.213289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab065b-74b0-4ddd-a403-bbf1de205173 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:24.219071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:24.220088 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.221287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.221767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] PATH_INFO: `/v3/projects/a9f7810d7b2046a1a574cadd97a35f86` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.221920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:24.223806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9f7810d7b2046a1a574cadd97a35f86)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:24.225929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:24.234457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab065b-74b0-4ddd-a403-bbf1de205173 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:24.242743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec37af-4517-46f5-9847-97b31423c585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:24.243054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec37af-4517-46f5-9847-97b31423c585 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.243458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec37af-4517-46f5-9847-97b31423c585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.244003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec37af-4517-46f5-9847-97b31423c585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.244747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec37af-4517-46f5-9847-97b31423c585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:24.248146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec37af-4517-46f5-9847-97b31423c585 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:24.264530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1061/2107] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:24.268771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec37af-4517-46f5-9847-97b31423c585 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:24.286036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec37af-4517-46f5-9847-97b31423c585 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:24.302806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9f7810d7b2046a1a574cadd97a35f86', 'request_id': 'req-0ede4555-463e-45d4-9ecf-1de91b88ab24'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.310435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9f7810d7b2046a1a574cadd97a35f86', 'request_id': 'req-0ede4555-463e-45d4-9ecf-1de91b88ab24'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.322259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9f7810d7b2046a1a574cadd97a35f86', 'request_id': 'req-0ede4555-463e-45d4-9ecf-1de91b88ab24'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.334648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:24.339670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] The token cache is being invalidate because project a9f7810d7b2046a1a574cadd97a35f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:24.340478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.342937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.344811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1047/2108] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:24 2022] DELETE /identity/v3/projects/a9f7810d7b2046a1a574cadd97a35f86 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:24.349413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:24.352381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:24.362286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed1378-dd8e-45a6-b47d-c694c06a7109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:24.363104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed1378-dd8e-45a6-b47d-c694c06a7109 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.363645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed1378-dd8e-45a6-b47d-c694c06a7109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.364077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed1378-dd8e-45a6-b47d-c694c06a7109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.364872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed1378-dd8e-45a6-b47d-c694c06a7109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:24.366669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed1378-dd8e-45a6-b47d-c694c06a7109 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:24.378009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1062/2109] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:24.382235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed1378-dd8e-45a6-b47d-c694c06a7109 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:24.384147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-03ed1378-dd8e-45a6-b47d-c694c06a7109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:24.396850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed1378-dd8e-45a6-b47d-c694c06a7109 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:24.401759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33e37-3b0b-4298-9fff-7e7dbff8eaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:24.402414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33e37-3b0b-4298-9fff-7e7dbff8eaad admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.402688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33e37-3b0b-4298-9fff-7e7dbff8eaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.402964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33e37-3b0b-4298-9fff-7e7dbff8eaad admin admin] PATH_INFO: `/v3/projects/2f5053a0e5c747298d2f64f9f9dacb3e/users/69c021849e0a494181595949aac5b50d/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.419484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead33e37-3b0b-4298-9fff-7e7dbff8eaad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5053a0e5c747298d2f64f9f9dacb3e, user_id=69c021849e0a494181595949aac5b50d, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:24.422290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead33e37-3b0b-4298-9fff-7e7dbff8eaad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:24.438233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1063/2110] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:24 2022] PUT /identity/v3/projects/2f5053a0e5c747298d2f64f9f9dacb3e/users/69c021849e0a494181595949aac5b50d/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:24.442187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede4555-463e-45d4-9ecf-1de91b88ab24 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:24.450531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91f3c9-42d8-4dcf-860f-6ac88f2ec497 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.450973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91f3c9-42d8-4dcf-860f-6ac88f2ec497 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.451342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91f3c9-42d8-4dcf-860f-6ac88f2ec497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.497473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7c91f3c9-42d8-4dcf-860f-6ac88f2ec497 None None] MFA Rules not processed for user `69c021849e0a494181595949aac5b50d`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:24.498794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7c91f3c9-42d8-4dcf-860f-6ac88f2ec497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:24.532175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:24.532810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.533119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.534165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] PATH_INFO: `/v3/users/5d78ca2180d34ad59beb3e361cd8e4bb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.535340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d78ca2180d34ad59beb3e361cd8e4bb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:24.537538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:24.609455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1064/2111] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:24.610226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91f3c9-42d8-4dcf-860f-6ac88f2ec497 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:24.612831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d78ca2180d34ad59beb3e361cd8e4bb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.613669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d78ca2180d34ad59beb3e361cd8e4bb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.622608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91f3c9-42d8-4dcf-860f-6ac88f2ec497 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:24.625641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d78ca2180d34ad59beb3e361cd8e4bb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.626733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d78ca2180d34ad59beb3e361cd8e4bb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.628257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdada50-6ac7-420b-b168-72ea4dc2d31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:24.628961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdada50-6ac7-420b-b168-72ea4dc2d31e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.629254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdada50-6ac7-420b-b168-72ea4dc2d31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.629525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdada50-6ac7-420b-b168-72ea4dc2d31e admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.630180 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdada50-6ac7-420b-b168-72ea4dc2d31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:24.632212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdada50-6ac7-420b-b168-72ea4dc2d31e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:24.645276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d78ca2180d34ad59beb3e361cd8e4bb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:24.672519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1048/2112] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:24 2022] DELETE /identity/v3/users/5d78ca2180d34ad59beb3e361cd8e4bb => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:24.687909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1065/2113] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:24 2022] POST /identity/v3/projects => generated 377 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:24.692037 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:24.694440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:24.771588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:24.833363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cf291-984d-443e-8983-30e653f72432 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:24.945998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f34959-192b-40eb-ac20-d994e7a0ee12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:24.946772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f34959-192b-40eb-ac20-d994e7a0ee12 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:24.947170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f34959-192b-40eb-ac20-d994e7a0ee12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:24.947538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f34959-192b-40eb-ac20-d994e7a0ee12 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:24.948477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f34959-192b-40eb-ac20-d994e7a0ee12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:24.951406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f34959-192b-40eb-ac20-d994e7a0ee12 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:24.977625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1049/2114] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:24 2022] POST /identity/v3/users => generated 295 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:24.982745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdada50-6ac7-420b-b168-72ea4dc2d31e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:24.994841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdada50-6ac7-420b-b168-72ea4dc2d31e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:25.000580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7e8b2-c1a4-454b-8b9f-b44fb403296f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:25.001446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e8b2-c1a4-454b-8b9f-b44fb403296f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.001838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e8b2-c1a4-454b-8b9f-b44fb403296f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.002198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e8b2-c1a4-454b-8b9f-b44fb403296f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.002935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7e8b2-c1a4-454b-8b9f-b44fb403296f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:25.004765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7e8b2-c1a4-454b-8b9f-b44fb403296f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:25.016693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1066/2115] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:25.021334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f34959-192b-40eb-ac20-d994e7a0ee12 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:25.030066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7e8b2-c1a4-454b-8b9f-b44fb403296f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:25.031788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-23b7e8b2-c1a4-454b-8b9f-b44fb403296f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:25.032310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f34959-192b-40eb-ac20-d994e7a0ee12 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:25.040737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e7d78-7d9a-4547-971d-3c173daae08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:25.041815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e7d78-7d9a-4547-971d-3c173daae08d admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.042107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e7d78-7d9a-4547-971d-3c173daae08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.042386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e7d78-7d9a-4547-971d-3c173daae08d admin admin] PATH_INFO: `/v3/projects/c20342b2e0c242f5a4668547bf9ca1e6/users/70da8335eed649cf8ff8d064f8fc3f9e/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.062733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7e8b2-c1a4-454b-8b9f-b44fb403296f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:25.064651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e7d78-7d9a-4547-971d-3c173daae08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20342b2e0c242f5a4668547bf9ca1e6, user_id=70da8335eed649cf8ff8d064f8fc3f9e, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:25.068531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e7d78-7d9a-4547-971d-3c173daae08d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:25.069448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:25.070155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.070450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.070779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] PATH_INFO: `/v3/projects/055fefe35dde4a8eb2752797e80a34cd` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.071950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=055fefe35dde4a8eb2752797e80a34cd)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:25.074143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:25.083682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1050/2116] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:25 2022] PUT /identity/v3/projects/c20342b2e0c242f5a4668547bf9ca1e6/users/70da8335eed649cf8ff8d064f8fc3f9e/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:25.092353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e7d78-7d9a-4547-971d-3c173daae08d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:25.098262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:25.137222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '055fefe35dde4a8eb2752797e80a34cd', 'request_id': 'req-bbfb86ae-d01d-4309-8915-f8be5e019c83'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.144914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '055fefe35dde4a8eb2752797e80a34cd', 'request_id': 'req-bbfb86ae-d01d-4309-8915-f8be5e019c83'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.154228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '055fefe35dde4a8eb2752797e80a34cd', 'request_id': 'req-bbfb86ae-d01d-4309-8915-f8be5e019c83'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.170271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] The token cache is being invalidate because project 055fefe35dde4a8eb2752797e80a34cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:25.170725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.171153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e7d78-7d9a-4547-971d-3c173daae08d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:25.171498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.173516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1067/2117] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:25 2022] DELETE /identity/v3/projects/055fefe35dde4a8eb2752797e80a34cd => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:25.177644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:25.178957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:25.217924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:25.236965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439797d-0bf2-430e-8ba9-0aae67dd5d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:25.237597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439797d-0bf2-430e-8ba9-0aae67dd5d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.237877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439797d-0bf2-430e-8ba9-0aae67dd5d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.238151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439797d-0bf2-430e-8ba9-0aae67dd5d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.238802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439797d-0bf2-430e-8ba9-0aae67dd5d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:25.240432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439797d-0bf2-430e-8ba9-0aae67dd5d1b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:25.248694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb86ae-d01d-4309-8915-f8be5e019c83 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:25.255279 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1051/2118] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:25.256330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439797d-0bf2-430e-8ba9-0aae67dd5d1b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:25.256761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:25.257386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.257665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.257939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] PATH_INFO: `/v3/users/303be1fec37a46b9a84c425930107bf2` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.258034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d439797d-0bf2-430e-8ba9-0aae67dd5d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:25.259066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=303be1fec37a46b9a84c425930107bf2)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:25.261048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:25.270412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439797d-0bf2-430e-8ba9-0aae67dd5d1b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:25.277065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecb164-0365-411b-9d1b-d9d5b34109b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:25.277742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecb164-0365-411b-9d1b-d9d5b34109b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.278024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecb164-0365-411b-9d1b-d9d5b34109b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.278305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecb164-0365-411b-9d1b-d9d5b34109b3 admin admin] PATH_INFO: `/v3/projects/c20342b2e0c242f5a4668547bf9ca1e6/users/70da8335eed649cf8ff8d064f8fc3f9e/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.280415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecb164-0365-411b-9d1b-d9d5b34109b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20342b2e0c242f5a4668547bf9ca1e6, user_id=70da8335eed649cf8ff8d064f8fc3f9e, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:25.283062 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecb164-0365-411b-9d1b-d9d5b34109b3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:25.294744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1052/2119] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:25 2022] PUT /identity/v3/projects/c20342b2e0c242f5a4668547bf9ca1e6/users/70da8335eed649cf8ff8d064f8fc3f9e/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:25.305247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735801b-2c12-41e1-90b6-e6ae2b4b1e78 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.305563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735801b-2c12-41e1-90b6-e6ae2b4b1e78 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.305843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735801b-2c12-41e1-90b6-e6ae2b4b1e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.314994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303be1fec37a46b9a84c425930107bf2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.325925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303be1fec37a46b9a84c425930107bf2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.326965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303be1fec37a46b9a84c425930107bf2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.327603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303be1fec37a46b9a84c425930107bf2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.336917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303be1fec37a46b9a84c425930107bf2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.338935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a735801b-2c12-41e1-90b6-e6ae2b4b1e78 None None] MFA Rules not processed for user `70da8335eed649cf8ff8d064f8fc3f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:25.340329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a735801b-2c12-41e1-90b6-e6ae2b4b1e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:25.362057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1068/2120] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:25 2022] DELETE /identity/v3/users/303be1fec37a46b9a84c425930107bf2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:25.417551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1053/2121] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:25.430576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:25.432082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:25.465082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:25.512061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717639b-d68f-4eae-beb8-600b3d279a6c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:25.516405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3ff3d-86a1-4855-a484-68d69a0f4ffb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:25.517019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3ff3d-86a1-4855-a484-68d69a0f4ffb service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.517298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3ff3d-86a1-4855-a484-68d69a0f4ffb service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.517572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3ff3d-86a1-4855-a484-68d69a0f4ffb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.519335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-88b3ff3d-86a1-4855-a484-68d69a0f4ffb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:25.609994 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3ff3d-86a1-4855-a484-68d69a0f4ffb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:25.611541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b3ff3d-86a1-4855-a484-68d69a0f4ffb service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:25.622976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1069/2122] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:25.665736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735801b-2c12-41e1-90b6-e6ae2b4b1e78 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:25.667396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a735801b-2c12-41e1-90b6-e6ae2b4b1e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:25.767198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735801b-2c12-41e1-90b6-e6ae2b4b1e78 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:25.843239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:25.843839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.844125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.844398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] PATH_INFO: `/v3/projects/62987c0e52cb45bd96e8a11d9b7346b8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.845638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62987c0e52cb45bd96e8a11d9b7346b8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:25.847590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:25.859249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:25.903533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62987c0e52cb45bd96e8a11d9b7346b8', 'request_id': 'req-a69bcdf2-167d-4997-8f3d-2b910fcb6672'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.910271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62987c0e52cb45bd96e8a11d9b7346b8', 'request_id': 'req-a69bcdf2-167d-4997-8f3d-2b910fcb6672'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.918320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62987c0e52cb45bd96e8a11d9b7346b8', 'request_id': 'req-a69bcdf2-167d-4997-8f3d-2b910fcb6672'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.931722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] The token cache is being invalidate because project 62987c0e52cb45bd96e8a11d9b7346b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:25.932044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.932755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a69bcdf2-167d-4997-8f3d-2b910fcb6672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:25.935796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1054/2123] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:25 2022] DELETE /identity/v3/projects/62987c0e52cb45bd96e8a11d9b7346b8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:25.941103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7396e-8bbb-40b9-80f9-74e5233e6a37 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:25.941488 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7396e-8bbb-40b9-80f9-74e5233e6a37 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:25.941765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7396e-8bbb-40b9-80f9-74e5233e6a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:25.963749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-62b7396e-8bbb-40b9-80f9-74e5233e6a37 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:25.965005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-62b7396e-8bbb-40b9-80f9-74e5233e6a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:26.032142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1070/2124] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:26.040740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4dda7-7ece-4998-a2ec-63df20c02ef2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:26.041031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4dda7-7ece-4998-a2ec-63df20c02ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:26.041308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4dda7-7ece-4998-a2ec-63df20c02ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:26.064589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e8a4dda7-7ece-4998-a2ec-63df20c02ef2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:26.064824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e8a4dda7-7ece-4998-a2ec-63df20c02ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:26.072861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1055/2125] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:26.077468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7396e-8bbb-40b9-80f9-74e5233e6a37 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:26.087449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7396e-8bbb-40b9-80f9-74e5233e6a37 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:26.092413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac23fc-30e0-429f-9d4c-e9d180aa06e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:26.093003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac23fc-30e0-429f-9d4c-e9d180aa06e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:26.093340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac23fc-30e0-429f-9d4c-e9d180aa06e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:26.093558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac23fc-30e0-429f-9d4c-e9d180aa06e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:26.094197 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac23fc-30e0-429f-9d4c-e9d180aa06e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:26.094498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac23fc-30e0-429f-9d4c-e9d180aa06e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:26.096124 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac23fc-30e0-429f-9d4c-e9d180aa06e4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:26.108586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1071/2126] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:22:26.113061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4dda7-7ece-4998-a2ec-63df20c02ef2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:26.123314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4dda7-7ece-4998-a2ec-63df20c02ef2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:26.128277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620e955-1ff5-416c-8a5e-b673bb18fce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:26.128881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620e955-1ff5-416c-8a5e-b673bb18fce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:26.129164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620e955-1ff5-416c-8a5e-b673bb18fce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:26.129435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620e955-1ff5-416c-8a5e-b673bb18fce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:26.130065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620e955-1ff5-416c-8a5e-b673bb18fce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:26.132011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620e955-1ff5-416c-8a5e-b673bb18fce0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:26.158683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1056/2127] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:26 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:22:26.163028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac23fc-30e0-429f-9d4c-e9d180aa06e4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:26.173536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac23fc-30e0-429f-9d4c-e9d180aa06e4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:26.229592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18628491-0923-483f-bd0a-df83c07964de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:26.230195 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628491-0923-483f-bd0a-df83c07964de admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:26.230475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628491-0923-483f-bd0a-df83c07964de admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:26.230746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18628491-0923-483f-bd0a-df83c07964de admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:26.231933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18628491-0923-483f-bd0a-df83c07964de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:26.233852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18628491-0923-483f-bd0a-df83c07964de admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:26.244040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:26.256504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1072/2128] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:26 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:26.260796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620e955-1ff5-416c-8a5e-b673bb18fce0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:26.271275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620e955-1ff5-416c-8a5e-b673bb18fce0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:26.276673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59c273-c819-46fc-97fa-c06b4933f16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:26.277276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59c273-c819-46fc-97fa-c06b4933f16b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:26.277556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59c273-c819-46fc-97fa-c06b4933f16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:26.277829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59c273-c819-46fc-97fa-c06b4933f16b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:26.278525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59c273-c819-46fc-97fa-c06b4933f16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:26.280161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59c273-c819-46fc-97fa-c06b4933f16b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:26.290622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1057/2129] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:26.295267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18628491-0923-483f-bd0a-df83c07964de admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:26.305648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18628491-0923-483f-bd0a-df83c07964de admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:26.310977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de9069-d669-4984-ac5c-c691ddf584fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:26.311578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de9069-d669-4984-ac5c-c691ddf584fb admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:26.311898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de9069-d669-4984-ac5c-c691ddf584fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:26.312176 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de9069-d669-4984-ac5c-c691ddf584fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:26.312819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de9069-d669-4984-ac5c-c691ddf584fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:26.314416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de9069-d669-4984-ac5c-c691ddf584fb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:26.325072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1073/2130] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:26.329142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59c273-c819-46fc-97fa-c06b4933f16b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:26.339196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59c273-c819-46fc-97fa-c06b4933f16b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:26.344409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc8961-7bf0-4df0-9e4b-fdfe1094160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:26.344995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc8961-7bf0-4df0-9e4b-fdfe1094160b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:26.345274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc8961-7bf0-4df0-9e4b-fdfe1094160b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:26.345545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc8961-7bf0-4df0-9e4b-fdfe1094160b admin admin] PATH_INFO: `/v3/projects/2f8142a083184e349a53858cb30d900d/users/769ebe8d7638498cb70ab1902c2d907d/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:26.361116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc8961-7bf0-4df0-9e4b-fdfe1094160b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8142a083184e349a53858cb30d900d, user_id=769ebe8d7638498cb70ab1902c2d907d, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:26.363827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc8961-7bf0-4df0-9e4b-fdfe1094160b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:26.378075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1058/2131] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:26 2022] PUT /identity/v3/projects/2f8142a083184e349a53858cb30d900d/users/769ebe8d7638498cb70ab1902c2d907d/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:26.390081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5357d6-27cc-4af8-88be-a24c67a07c9d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:26.390363 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5357d6-27cc-4af8-88be-a24c67a07c9d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:26.390661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5357d6-27cc-4af8-88be-a24c67a07c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:26.421310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bd5357d6-27cc-4af8-88be-a24c67a07c9d None None] MFA Rules not processed for user `769ebe8d7638498cb70ab1902c2d907d`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:26.422624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bd5357d6-27cc-4af8-88be-a24c67a07c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:26.487597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1074/2132] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:26.498992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc8961-7bf0-4df0-9e4b-fdfe1094160b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:26.500695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8ecc8961-7bf0-4df0-9e4b-fdfe1094160b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:26.557735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc8961-7bf0-4df0-9e4b-fdfe1094160b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:26.561528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073a59f-25ff-4706-af9e-d617151198bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:26.562129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073a59f-25ff-4706-af9e-d617151198bb service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:26.562403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073a59f-25ff-4706-af9e-d617151198bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:26.562672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073a59f-25ff-4706-af9e-d617151198bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:26.570880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073a59f-25ff-4706-af9e-d617151198bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:26.572322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073a59f-25ff-4706-af9e-d617151198bb service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:26.580962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1059/2133] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:31.497406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5357d6-27cc-4af8-88be-a24c67a07c9d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:31.506943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5357d6-27cc-4af8-88be-a24c67a07c9d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:31.510511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:31.511116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:31.511396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:31.511669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:31.513463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:31.574506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:31.576031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:31.585405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1075/2134] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:32.264550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073a59f-25ff-4706-af9e-d617151198bb service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:32.273688 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073a59f-25ff-4706-af9e-d617151198bb service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:32.277493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066ed10-b3c3-4010-9b4a-fb954de487d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:32.278606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066ed10-b3c3-4010-9b4a-fb954de487d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:32.278881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066ed10-b3c3-4010-9b4a-fb954de487d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:32.279151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066ed10-b3c3-4010-9b4a-fb954de487d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:32.280939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c066ed10-b3c3-4010-9b4a-fb954de487d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:32.313371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:32.360843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066ed10-b3c3-4010-9b4a-fb954de487d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:32.362535 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066ed10-b3c3-4010-9b4a-fb954de487d4 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:32.373888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1060/2135] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:37.959415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:37.962170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:38.042112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a87b40-dbc8-4d26-aea2-b60d7e24674f service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:38.047393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:38.048228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:38.048576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:38.048918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] PATH_INFO: `/v3/users/1d73d5c579d94f428228e3dd4f231118` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:38.050351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d73d5c579d94f428228e3dd4f231118)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:38.053123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:38.094592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=1,keystone:SELECT=98,keystone:DELETE=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:38.149752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d73d5c579d94f428228e3dd4f231118'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.158362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d73d5c579d94f428228e3dd4f231118'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.159100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d73d5c579d94f428228e3dd4f231118'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.159730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d73d5c579d94f428228e3dd4f231118'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.167397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d73d5c579d94f428228e3dd4f231118'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.186562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1076/2136] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:37 2022] DELETE /identity/v3/users/1d73d5c579d94f428228e3dd4f231118 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:38.191117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4965d-b1ba-443b-b444-a10880455d02 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:38.191591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4965d-b1ba-443b-b444-a10880455d02 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:38.191714 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4965d-b1ba-443b-b444-a10880455d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:38.212479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e2c4965d-b1ba-443b-b444-a10880455d02 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:38.213731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2c4965d-b1ba-443b-b444-a10880455d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:38.282293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1061/2137] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:38.291782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:38.293271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:38.352666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fa3f4-44ef-43a6-8fb3-dc28ae117e49 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:38.357925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47ca07-718b-4df7-853b-410392cd23f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:38.358697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47ca07-718b-4df7-853b-410392cd23f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:38.359227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47ca07-718b-4df7-853b-410392cd23f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:38.359509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47ca07-718b-4df7-853b-410392cd23f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:38.370782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47ca07-718b-4df7-853b-410392cd23f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:38.373091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47ca07-718b-4df7-853b-410392cd23f6 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:38.383942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1077/2138] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:38.688738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4965d-b1ba-443b-b444-a10880455d02 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:38.690236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e2c4965d-b1ba-443b-b444-a10880455d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:38.704804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4965d-b1ba-443b-b444-a10880455d02 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:38.713442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:38.714438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:38.714642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:38.715030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] PATH_INFO: `/v3/projects/01322137ba0b464aad5a1b1d9499e8ed` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:38.717729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] RBAC: Authorizing `identity:delete_project(project_id=01322137ba0b464aad5a1b1d9499e8ed)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:38.719973 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:38.769042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01322137ba0b464aad5a1b1d9499e8ed', 'request_id': 'req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.775169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01322137ba0b464aad5a1b1d9499e8ed', 'request_id': 'req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.783505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01322137ba0b464aad5a1b1d9499e8ed', 'request_id': 'req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.794442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] The token cache is being invalidate because project 01322137ba0b464aad5a1b1d9499e8ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:38.794734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.795383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.796891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1062/2139] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:38 2022] DELETE /identity/v3/projects/01322137ba0b464aad5a1b1d9499e8ed => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:38.801047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47ca07-718b-4df7-853b-410392cd23f6 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:38.810747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47ca07-718b-4df7-853b-410392cd23f6 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:38.866954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:38.867581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:38.867890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:38.868165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] PATH_INFO: `/v3/users/e471de98af6545a3ba2f804be569cfc7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:38.869274 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e471de98af6545a3ba2f804be569cfc7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:38.871206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:38.910948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:38.921917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e471de98af6545a3ba2f804be569cfc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.929155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e471de98af6545a3ba2f804be569cfc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.929771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e471de98af6545a3ba2f804be569cfc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.930378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e471de98af6545a3ba2f804be569cfc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.937386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-b4be1113-52d6-4248-a5eb-5fef26886767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e471de98af6545a3ba2f804be569cfc7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:38.955052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1078/2140] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:38 2022] DELETE /identity/v3/users/e471de98af6545a3ba2f804be569cfc7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:39.234138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:39.235635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:39.260316 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fc939-82e8-47a7-97f1-2eb9d4a3f76f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:39.320631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:39.325101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:39.326507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.327266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.332129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] PATH_INFO: `/v3/projects/f28391e42e2a4fe8b216495e3d6c68b7` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.339506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=f28391e42e2a4fe8b216495e3d6c68b7)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:39.341525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:39.388981 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f28391e42e2a4fe8b216495e3d6c68b7', 'request_id': 'req-2ac789ee-30b3-4d80-a12e-b07ea1401dea'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:39.395101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f28391e42e2a4fe8b216495e3d6c68b7', 'request_id': 'req-2ac789ee-30b3-4d80-a12e-b07ea1401dea'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:39.402099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f28391e42e2a4fe8b216495e3d6c68b7', 'request_id': 'req-2ac789ee-30b3-4d80-a12e-b07ea1401dea'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:39.412672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] The token cache is being invalidate because project f28391e42e2a4fe8b216495e3d6c68b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:39.412962 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:39.413586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-2ac789ee-30b3-4d80-a12e-b07ea1401dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:39.415068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1063/2141] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:39 2022] DELETE /identity/v3/projects/f28391e42e2a4fe8b216495e3d6c68b7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:39.421704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c0fa3-36a3-4cdc-87e3-dd674eae0be0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.422193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c0fa3-36a3-4cdc-87e3-dd674eae0be0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.422516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c0fa3-36a3-4cdc-87e3-dd674eae0be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.442442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5e4c0fa3-36a3-4cdc-87e3-dd674eae0be0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:39.443638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5e4c0fa3-36a3-4cdc-87e3-dd674eae0be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:39.502937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1079/2142] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:39.510896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49221b7f-0c5c-44b5-b703-4795086a3d22 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.511186 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49221b7f-0c5c-44b5-b703-4795086a3d22 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.511466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49221b7f-0c5c-44b5-b703-4795086a3d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.530937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-49221b7f-0c5c-44b5-b703-4795086a3d22 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:39.532194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-49221b7f-0c5c-44b5-b703-4795086a3d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:39.540513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1064/2143] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:39.545442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c0fa3-36a3-4cdc-87e3-dd674eae0be0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:39.555319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c0fa3-36a3-4cdc-87e3-dd674eae0be0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:39.559907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337018f6-160f-417c-9153-72ca937f3162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:39.561105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337018f6-160f-417c-9153-72ca937f3162 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.561387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337018f6-160f-417c-9153-72ca937f3162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.561660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337018f6-160f-417c-9153-72ca937f3162 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.562334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337018f6-160f-417c-9153-72ca937f3162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:39.562637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337018f6-160f-417c-9153-72ca937f3162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:39.564265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337018f6-160f-417c-9153-72ca937f3162 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:39.575660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1080/2144] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:22:39.580025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49221b7f-0c5c-44b5-b703-4795086a3d22 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:39.589466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49221b7f-0c5c-44b5-b703-4795086a3d22 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:39.594034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503355a7-ebf9-4f28-8335-c735c0f2c3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:39.594626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503355a7-ebf9-4f28-8335-c735c0f2c3ba admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.594900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503355a7-ebf9-4f28-8335-c735c0f2c3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.595165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503355a7-ebf9-4f28-8335-c735c0f2c3ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.595816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503355a7-ebf9-4f28-8335-c735c0f2c3ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:39.597695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503355a7-ebf9-4f28-8335-c735c0f2c3ba admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:39.626607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1065/2145] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:39 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:39.630840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337018f6-160f-417c-9153-72ca937f3162 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:39.641370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337018f6-160f-417c-9153-72ca937f3162 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:39.698817 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b6127-c11f-4017-aca1-0dc2a7f05e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:39.699435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b6127-c11f-4017-aca1-0dc2a7f05e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.699721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b6127-c11f-4017-aca1-0dc2a7f05e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.700040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b6127-c11f-4017-aca1-0dc2a7f05e24 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.700685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b6127-c11f-4017-aca1-0dc2a7f05e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:39.702610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b6127-c11f-4017-aca1-0dc2a7f05e24 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:39.724990 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1081/2146] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:39 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:39.729166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503355a7-ebf9-4f28-8335-c735c0f2c3ba admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:39.740111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503355a7-ebf9-4f28-8335-c735c0f2c3ba admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:39.744914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac495c1-c5e9-4297-af68-73f61ba84d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:39.745632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac495c1-c5e9-4297-af68-73f61ba84d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.746012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac495c1-c5e9-4297-af68-73f61ba84d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.746388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac495c1-c5e9-4297-af68-73f61ba84d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.747265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac495c1-c5e9-4297-af68-73f61ba84d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:39.749599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac495c1-c5e9-4297-af68-73f61ba84d77 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:39.762728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1066/2147] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:39.764824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b6127-c11f-4017-aca1-0dc2a7f05e24 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:39.775350 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b6127-c11f-4017-aca1-0dc2a7f05e24 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:39.780453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107d93e-533b-49e3-af07-05e36464e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:39.781067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107d93e-533b-49e3-af07-05e36464e6aa admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.781409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107d93e-533b-49e3-af07-05e36464e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.781779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107d93e-533b-49e3-af07-05e36464e6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.782339 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107d93e-533b-49e3-af07-05e36464e6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:39.784170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107d93e-533b-49e3-af07-05e36464e6aa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:39.798167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1082/2148] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:39.800566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac495c1-c5e9-4297-af68-73f61ba84d77 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:39.811333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac495c1-c5e9-4297-af68-73f61ba84d77 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:39.815974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76bee6-df31-4138-9947-ae5dbae8c71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:39.816614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76bee6-df31-4138-9947-ae5dbae8c71b admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.816896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76bee6-df31-4138-9947-ae5dbae8c71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.817172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76bee6-df31-4138-9947-ae5dbae8c71b admin admin] PATH_INFO: `/v3/projects/ab041a7d1acb4e37a6dfc282f0d33bef/users/48d24d7c96224fddb3c6602fc41b2362/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.831861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76bee6-df31-4138-9947-ae5dbae8c71b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab041a7d1acb4e37a6dfc282f0d33bef, user_id=48d24d7c96224fddb3c6602fc41b2362, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:39.834520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76bee6-df31-4138-9947-ae5dbae8c71b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:39.848329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1067/2149] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:39 2022] PUT /identity/v3/projects/ab041a7d1acb4e37a6dfc282f0d33bef/users/48d24d7c96224fddb3c6602fc41b2362/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:39.860281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1f416-cf98-48f5-b6e3-d3b68bfb4d7e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:39.860559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1f416-cf98-48f5-b6e3-d3b68bfb4d7e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:39.860833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1f416-cf98-48f5-b6e3-d3b68bfb4d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:39.893679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f6a1f416-cf98-48f5-b6e3-d3b68bfb4d7e None None] MFA Rules not processed for user `48d24d7c96224fddb3c6602fc41b2362`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:39.894907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f6a1f416-cf98-48f5-b6e3-d3b68bfb4d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:39.901095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:39.967162 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1083/2150] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:39.971667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76bee6-df31-4138-9947-ae5dbae8c71b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:40.032503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76bee6-df31-4138-9947-ae5dbae8c71b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:40.037263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e2e96-ad4d-4c5a-8c67-9cbf1825395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:40.037860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e2e96-ad4d-4c5a-8c67-9cbf1825395e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:40.038128 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e2e96-ad4d-4c5a-8c67-9cbf1825395e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:40.038396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e2e96-ad4d-4c5a-8c67-9cbf1825395e admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:40.039045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e2e96-ad4d-4c5a-8c67-9cbf1825395e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:40.040974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e2e96-ad4d-4c5a-8c67-9cbf1825395e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:40.062693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:40.070900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1068/2151] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:39 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:40.075096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1f416-cf98-48f5-b6e3-d3b68bfb4d7e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:40.086015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1f416-cf98-48f5-b6e3-d3b68bfb4d7e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:40.141423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f89dbc-162d-40ae-b9fc-b12e52ed325b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:40.142027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f89dbc-162d-40ae-b9fc-b12e52ed325b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:40.142307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f89dbc-162d-40ae-b9fc-b12e52ed325b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:40.142579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f89dbc-162d-40ae-b9fc-b12e52ed325b admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:40.143203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f89dbc-162d-40ae-b9fc-b12e52ed325b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:40.145150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f89dbc-162d-40ae-b9fc-b12e52ed325b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:40.167021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1084/2152] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:40 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:40.171726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e2e96-ad4d-4c5a-8c67-9cbf1825395e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:40.181700 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e2e96-ad4d-4c5a-8c67-9cbf1825395e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:40.186516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263cd19-3ea8-4a4c-acbe-76ad96d9e3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:40.187112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263cd19-3ea8-4a4c-acbe-76ad96d9e3dc admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:40.187384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263cd19-3ea8-4a4c-acbe-76ad96d9e3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:40.187651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263cd19-3ea8-4a4c-acbe-76ad96d9e3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:40.188317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263cd19-3ea8-4a4c-acbe-76ad96d9e3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:40.189909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263cd19-3ea8-4a4c-acbe-76ad96d9e3dc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:40.202078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1069/2153] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:40.206670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f89dbc-162d-40ae-b9fc-b12e52ed325b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:40.217580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f89dbc-162d-40ae-b9fc-b12e52ed325b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:40.222301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58241d16-f4c2-434c-bc80-308da465aebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:40.222952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58241d16-f4c2-434c-bc80-308da465aebf admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:40.223235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58241d16-f4c2-434c-bc80-308da465aebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:40.223511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58241d16-f4c2-434c-bc80-308da465aebf admin admin] PATH_INFO: `/v3/projects/db3a2c2bf6fe44e78fc956e5723ce11e/users/e02642ddae224ba796712c196af26657/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:40.238244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58241d16-f4c2-434c-bc80-308da465aebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3a2c2bf6fe44e78fc956e5723ce11e, user_id=e02642ddae224ba796712c196af26657, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:40.240949 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58241d16-f4c2-434c-bc80-308da465aebf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:40.253403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1085/2154] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:40 2022] PUT /identity/v3/projects/db3a2c2bf6fe44e78fc956e5723ce11e/users/e02642ddae224ba796712c196af26657/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:40.257750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263cd19-3ea8-4a4c-acbe-76ad96d9e3dc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:40.268493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263cd19-3ea8-4a4c-acbe-76ad96d9e3dc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:40.323939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25c634-8d06-4480-8ece-3e125f95702f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:40.324652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25c634-8d06-4480-8ece-3e125f95702f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:40.324936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25c634-8d06-4480-8ece-3e125f95702f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:40.325209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25c634-8d06-4480-8ece-3e125f95702f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:40.326337 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25c634-8d06-4480-8ece-3e125f95702f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:40.327986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25c634-8d06-4480-8ece-3e125f95702f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:40.338691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1070/2155] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:40.344508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58241d16-f4c2-434c-bc80-308da465aebf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:40.356092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58241d16-f4c2-434c-bc80-308da465aebf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:40.360228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f1a30-22f2-4fe5-977d-20c406396964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:40.360978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f1a30-22f2-4fe5-977d-20c406396964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:40.361358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f1a30-22f2-4fe5-977d-20c406396964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:40.361695 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f1a30-22f2-4fe5-977d-20c406396964 admin admin] PATH_INFO: `/v3/projects/db3a2c2bf6fe44e78fc956e5723ce11e/users/e02642ddae224ba796712c196af26657/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:40.363824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f1a30-22f2-4fe5-977d-20c406396964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3a2c2bf6fe44e78fc956e5723ce11e, user_id=e02642ddae224ba796712c196af26657, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:40.366690 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f1a30-22f2-4fe5-977d-20c406396964 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:40.377901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1086/2156] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:40 2022] PUT /identity/v3/projects/db3a2c2bf6fe44e78fc956e5723ce11e/users/e02642ddae224ba796712c196af26657/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:40.393517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf646c-dc80-4b40-ba41-abf47ace3252 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:40.393815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf646c-dc80-4b40-ba41-abf47ace3252 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:40.394091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf646c-dc80-4b40-ba41-abf47ace3252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:40.423275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-eccf646c-dc80-4b40-ba41-abf47ace3252 None None] MFA Rules not processed for user `e02642ddae224ba796712c196af26657`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:40.425174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eccf646c-dc80-4b40-ba41-abf47ace3252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:40.501069 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1071/2157] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:40.513640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f1a30-22f2-4fe5-977d-20c406396964 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:40.515811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f82f1a30-22f2-4fe5-977d-20c406396964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:40.584233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f1a30-22f2-4fe5-977d-20c406396964 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:40.587985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf41af-fa15-44fb-a486-41dde5399acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:40.588597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf41af-fa15-44fb-a486-41dde5399acc service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:40.588875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf41af-fa15-44fb-a486-41dde5399acc service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:40.589147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf41af-fa15-44fb-a486-41dde5399acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:40.598347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf41af-fa15-44fb-a486-41dde5399acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:40.599814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf41af-fa15-44fb-a486-41dde5399acc service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:40.609928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1087/2158] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:41.263633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf646c-dc80-4b40-ba41-abf47ace3252 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:41.332905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf646c-dc80-4b40-ba41-abf47ace3252 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:41.337881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:41.338527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:41.338809 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:41.339086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] PATH_INFO: `/v3/users/48d24d7c96224fddb3c6602fc41b2362` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:41.340493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d24d7c96224fddb3c6602fc41b2362)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:41.342964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:41.356311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:41.390963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d24d7c96224fddb3c6602fc41b2362'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:41.391617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d24d7c96224fddb3c6602fc41b2362'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:41.398979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d24d7c96224fddb3c6602fc41b2362'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:41.399762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d24d7c96224fddb3c6602fc41b2362'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:41.408331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d24d7c96224fddb3c6602fc41b2362'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:41.425928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1072/2159] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:41 2022] DELETE /identity/v3/users/48d24d7c96224fddb3c6602fc41b2362 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:41.430584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa9f5d-5be8-43f2-8e05-4f06ab99f25c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:41.431053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa9f5d-5be8-43f2-8e05-4f06ab99f25c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:41.431412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa9f5d-5be8-43f2-8e05-4f06ab99f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:41.439270 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:41.454421 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-43aa9f5d-5be8-43f2-8e05-4f06ab99f25c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:41.455628 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-43aa9f5d-5be8-43f2-8e05-4f06ab99f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:41.518678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1088/2160] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:41.531028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:41.533036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:41.601694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5d842-c4c7-4c10-9388-ad70bc4836bb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:41.605329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d9783-af98-4737-929e-50e7041500ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:41.605931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d9783-af98-4737-929e-50e7041500ff service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:41.606207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d9783-af98-4737-929e-50e7041500ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:41.606477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d9783-af98-4737-929e-50e7041500ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:41.616707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d9783-af98-4737-929e-50e7041500ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:41.618145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d9783-af98-4737-929e-50e7041500ff service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:41.629167 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1073/2161] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:41.919650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa9f5d-5be8-43f2-8e05-4f06ab99f25c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:41.921131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-43aa9f5d-5be8-43f2-8e05-4f06ab99f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:41.934381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa9f5d-5be8-43f2-8e05-4f06ab99f25c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:41.939350 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:41.940023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:41.940772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:41.940772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] PATH_INFO: `/v3/projects/ab041a7d1acb4e37a6dfc282f0d33bef` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:41.941715 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab041a7d1acb4e37a6dfc282f0d33bef)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:41.943630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:41.991320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab041a7d1acb4e37a6dfc282f0d33bef', 'request_id': 'req-9b78b91c-c183-4b3b-aea1-67117e0018ea'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:41.997388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab041a7d1acb4e37a6dfc282f0d33bef', 'request_id': 'req-9b78b91c-c183-4b3b-aea1-67117e0018ea'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.004992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab041a7d1acb4e37a6dfc282f0d33bef', 'request_id': 'req-9b78b91c-c183-4b3b-aea1-67117e0018ea'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.016733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] The token cache is being invalidate because project ab041a7d1acb4e37a6dfc282f0d33bef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:42.017014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.017643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.019103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1089/2162] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:41 2022] DELETE /identity/v3/projects/ab041a7d1acb4e37a6dfc282f0d33bef => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:42.023131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d9783-af98-4737-929e-50e7041500ff service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:42.034162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d9783-af98-4737-929e-50e7041500ff service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:42.097049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:42.097715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:42.097992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:42.098270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] PATH_INFO: `/v3/users/e02642ddae224ba796712c196af26657` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:42.099409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e02642ddae224ba796712c196af26657)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:42.101583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:42.144883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:42.157354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e02642ddae224ba796712c196af26657'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.158024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e02642ddae224ba796712c196af26657'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.165285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e02642ddae224ba796712c196af26657'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.165890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e02642ddae224ba796712c196af26657'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.173247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-6faa9976-3dcb-4ce1-930e-c468b7c14666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e02642ddae224ba796712c196af26657'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.192801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1074/2163] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:42 2022] DELETE /identity/v3/users/e02642ddae224ba796712c196af26657 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:42.520218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:42.521686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:42.535836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78b91c-c183-4b3b-aea1-67117e0018ea admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:42.595951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13300524-14aa-456d-9337-60867edb912b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:42.596583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13300524-14aa-456d-9337-60867edb912b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:42.596856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13300524-14aa-456d-9337-60867edb912b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:42.597123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13300524-14aa-456d-9337-60867edb912b admin admin] PATH_INFO: `/v3/projects/db3a2c2bf6fe44e78fc956e5723ce11e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:42.598245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13300524-14aa-456d-9337-60867edb912b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3a2c2bf6fe44e78fc956e5723ce11e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:42.600186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13300524-14aa-456d-9337-60867edb912b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:42.633996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:42.651163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-13300524-14aa-456d-9337-60867edb912b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3a2c2bf6fe44e78fc956e5723ce11e', 'request_id': 'req-13300524-14aa-456d-9337-60867edb912b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.657255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-13300524-14aa-456d-9337-60867edb912b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3a2c2bf6fe44e78fc956e5723ce11e', 'request_id': 'req-13300524-14aa-456d-9337-60867edb912b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.667151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-13300524-14aa-456d-9337-60867edb912b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3a2c2bf6fe44e78fc956e5723ce11e', 'request_id': 'req-13300524-14aa-456d-9337-60867edb912b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.679156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-13300524-14aa-456d-9337-60867edb912b admin admin] The token cache is being invalidate because project db3a2c2bf6fe44e78fc956e5723ce11e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:42.679459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-13300524-14aa-456d-9337-60867edb912b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.680111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-13300524-14aa-456d-9337-60867edb912b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:42.681642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1090/2164] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:42 2022] DELETE /identity/v3/projects/db3a2c2bf6fe44e78fc956e5723ce11e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:42.686965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f529f07-0361-46b5-ab87-16f5fbe9d59e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:42.687444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f529f07-0361-46b5-ab87-16f5fbe9d59e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:42.687534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f529f07-0361-46b5-ab87-16f5fbe9d59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:42.712118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9f529f07-0361-46b5-ab87-16f5fbe9d59e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:42.713619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9f529f07-0361-46b5-ab87-16f5fbe9d59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:42.778964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1075/2165] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:42.788011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392920af-85e3-42ad-90fd-c2d4508ec646 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:42.788298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392920af-85e3-42ad-90fd-c2d4508ec646 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:42.788570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392920af-85e3-42ad-90fd-c2d4508ec646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:42.811331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-392920af-85e3-42ad-90fd-c2d4508ec646 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:42.812581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-392920af-85e3-42ad-90fd-c2d4508ec646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:42.821566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1091/2166] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:42.825626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f529f07-0361-46b5-ab87-16f5fbe9d59e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:42.835971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f529f07-0361-46b5-ab87-16f5fbe9d59e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:42.841805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416641f1-b77c-4f25-ab0c-9ebc9cd0c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:42.842492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416641f1-b77c-4f25-ab0c-9ebc9cd0c8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:42.842838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416641f1-b77c-4f25-ab0c-9ebc9cd0c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:42.843179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416641f1-b77c-4f25-ab0c-9ebc9cd0c8f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:42.843892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416641f1-b77c-4f25-ab0c-9ebc9cd0c8f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:42.844203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416641f1-b77c-4f25-ab0c-9ebc9cd0c8f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:42.845927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416641f1-b77c-4f25-ab0c-9ebc9cd0c8f3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:42.858481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1076/2167] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:22:42.862801 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392920af-85e3-42ad-90fd-c2d4508ec646 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:42.871903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392920af-85e3-42ad-90fd-c2d4508ec646 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:42.878675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06559f83-7447-4e8c-8aee-e0950c2dc731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:42.879288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06559f83-7447-4e8c-8aee-e0950c2dc731 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:42.879564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06559f83-7447-4e8c-8aee-e0950c2dc731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:42.879863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06559f83-7447-4e8c-8aee-e0950c2dc731 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:42.880511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06559f83-7447-4e8c-8aee-e0950c2dc731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:42.882414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06559f83-7447-4e8c-8aee-e0950c2dc731 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:42.919449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1092/2168] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:42 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:42.925193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416641f1-b77c-4f25-ab0c-9ebc9cd0c8f3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:42.936244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416641f1-b77c-4f25-ab0c-9ebc9cd0c8f3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:42.994446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c797204-1409-4676-966b-c6e0491bba3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:42.995218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c797204-1409-4676-966b-c6e0491bba3b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:42.995564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c797204-1409-4676-966b-c6e0491bba3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:42.995976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c797204-1409-4676-966b-c6e0491bba3b admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:42.996672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c797204-1409-4676-966b-c6e0491bba3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:42.998828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c797204-1409-4676-966b-c6e0491bba3b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.020655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1077/2169] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:42 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:43.024781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06559f83-7447-4e8c-8aee-e0950c2dc731 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.035799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06559f83-7447-4e8c-8aee-e0950c2dc731 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.040475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bcf28-2593-48ca-bfd0-1646df189c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.041070 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bcf28-2593-48ca-bfd0-1646df189c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.041350 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bcf28-2593-48ca-bfd0-1646df189c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.041624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bcf28-2593-48ca-bfd0-1646df189c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.042257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bcf28-2593-48ca-bfd0-1646df189c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.043918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bcf28-2593-48ca-bfd0-1646df189c5e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.054465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1093/2170] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:43.058417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c797204-1409-4676-966b-c6e0491bba3b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.068997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c797204-1409-4676-966b-c6e0491bba3b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.074034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf0b30-0847-4115-95b2-ba02a27b12d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.074700 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf0b30-0847-4115-95b2-ba02a27b12d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.075008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf0b30-0847-4115-95b2-ba02a27b12d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.075308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf0b30-0847-4115-95b2-ba02a27b12d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.076086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf0b30-0847-4115-95b2-ba02a27b12d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.077782 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf0b30-0847-4115-95b2-ba02a27b12d7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.089061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1078/2171] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:43.094321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bcf28-2593-48ca-bfd0-1646df189c5e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.107506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bcf28-2593-48ca-bfd0-1646df189c5e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.112349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f7105-20e3-47f0-9c45-d46833af86ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.112999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f7105-20e3-47f0-9c45-d46833af86ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.113282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f7105-20e3-47f0-9c45-d46833af86ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.113559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f7105-20e3-47f0-9c45-d46833af86ee admin admin] PATH_INFO: `/v3/projects/b6da3a04825542218ee1a25add941dd1/users/a341379eb3ef4b31b367cbce41f3d566/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.128348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f7105-20e3-47f0-9c45-d46833af86ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6da3a04825542218ee1a25add941dd1, user_id=a341379eb3ef4b31b367cbce41f3d566, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.132269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f7105-20e3-47f0-9c45-d46833af86ee admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.147648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1094/2172] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:43 2022] PUT /identity/v3/projects/b6da3a04825542218ee1a25add941dd1/users/a341379eb3ef4b31b367cbce41f3d566/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:43.158298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acc76d-bc9c-4ada-a30a-a96bb40d7820 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.158581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acc76d-bc9c-4ada-a30a-a96bb40d7820 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.158853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acc76d-bc9c-4ada-a30a-a96bb40d7820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.189043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-22acc76d-bc9c-4ada-a30a-a96bb40d7820 None None] MFA Rules not processed for user `a341379eb3ef4b31b367cbce41f3d566`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:43.191807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-22acc76d-bc9c-4ada-a30a-a96bb40d7820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:43.202400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:43.282647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1079/2173] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:43.287108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f7105-20e3-47f0-9c45-d46833af86ee admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.356462 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f7105-20e3-47f0-9c45-d46833af86ee admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.361648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc89b6-e739-4fe3-89d4-aca6097dac2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.362352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc89b6-e739-4fe3-89d4-aca6097dac2b admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.362633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc89b6-e739-4fe3-89d4-aca6097dac2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.363039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc89b6-e739-4fe3-89d4-aca6097dac2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.363878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc89b6-e739-4fe3-89d4-aca6097dac2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.365854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc89b6-e739-4fe3-89d4-aca6097dac2b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.397755 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1095/2174] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:43 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:43.401890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acc76d-bc9c-4ada-a30a-a96bb40d7820 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.413467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acc76d-bc9c-4ada-a30a-a96bb40d7820 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.481601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14231b2-30a6-4769-94ff-8e7b8d7d48fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.482244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14231b2-30a6-4769-94ff-8e7b8d7d48fc admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.482526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14231b2-30a6-4769-94ff-8e7b8d7d48fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.482797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14231b2-30a6-4769-94ff-8e7b8d7d48fc admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.483545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14231b2-30a6-4769-94ff-8e7b8d7d48fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.485587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14231b2-30a6-4769-94ff-8e7b8d7d48fc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.507622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1080/2175] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:43 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:43.511470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc89b6-e739-4fe3-89d4-aca6097dac2b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.522772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc89b6-e739-4fe3-89d4-aca6097dac2b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.528002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685cf3d-ad0a-40c3-8996-47b9af8f2c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.529487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685cf3d-ad0a-40c3-8996-47b9af8f2c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.529766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685cf3d-ad0a-40c3-8996-47b9af8f2c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.530041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2685cf3d-ad0a-40c3-8996-47b9af8f2c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.530890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685cf3d-ad0a-40c3-8996-47b9af8f2c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.532658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2685cf3d-ad0a-40c3-8996-47b9af8f2c99 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.544716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1096/2176] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:43.548880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14231b2-30a6-4769-94ff-8e7b8d7d48fc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.558810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14231b2-30a6-4769-94ff-8e7b8d7d48fc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.564624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75e5b3-fc89-4e3d-a0f2-2b941e45bfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.565274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e5b3-fc89-4e3d-a0f2-2b941e45bfb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.565614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e5b3-fc89-4e3d-a0f2-2b941e45bfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.565823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e5b3-fc89-4e3d-a0f2-2b941e45bfb0 admin admin] PATH_INFO: `/v3/projects/9433da7f2dab407c816e7dfccb6ba9dd/users/6bebdfc80ec642e59de13bc670800e2d/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.582218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75e5b3-fc89-4e3d-a0f2-2b941e45bfb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9433da7f2dab407c816e7dfccb6ba9dd, user_id=6bebdfc80ec642e59de13bc670800e2d, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.585656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75e5b3-fc89-4e3d-a0f2-2b941e45bfb0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.598407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1081/2177] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:43 2022] PUT /identity/v3/projects/9433da7f2dab407c816e7dfccb6ba9dd/users/6bebdfc80ec642e59de13bc670800e2d/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:43.602375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685cf3d-ad0a-40c3-8996-47b9af8f2c99 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.612297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2685cf3d-ad0a-40c3-8996-47b9af8f2c99 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.636156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:43.674227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a27d7-2be8-429a-b796-976eaa72626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.674837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a27d7-2be8-429a-b796-976eaa72626c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.675121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a27d7-2be8-429a-b796-976eaa72626c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.675398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a27d7-2be8-429a-b796-976eaa72626c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.676075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a27d7-2be8-429a-b796-976eaa72626c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.677677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a27d7-2be8-429a-b796-976eaa72626c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.688172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1097/2178] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:43.692436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75e5b3-fc89-4e3d-a0f2-2b941e45bfb0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.703323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75e5b3-fc89-4e3d-a0f2-2b941e45bfb0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.708134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea626cee-0f06-4624-bad4-d1e4d885401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.708774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea626cee-0f06-4624-bad4-d1e4d885401e admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.709050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea626cee-0f06-4624-bad4-d1e4d885401e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.709323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea626cee-0f06-4624-bad4-d1e4d885401e admin admin] PATH_INFO: `/v3/projects/9433da7f2dab407c816e7dfccb6ba9dd/users/6bebdfc80ec642e59de13bc670800e2d/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.711200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea626cee-0f06-4624-bad4-d1e4d885401e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9433da7f2dab407c816e7dfccb6ba9dd, user_id=6bebdfc80ec642e59de13bc670800e2d, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.713909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea626cee-0f06-4624-bad4-d1e4d885401e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.723896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1082/2179] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:43 2022] PUT /identity/v3/projects/9433da7f2dab407c816e7dfccb6ba9dd/users/6bebdfc80ec642e59de13bc670800e2d/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:43.734022 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536009b9-cd27-4280-ba31-935a3441e5ad None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.734391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536009b9-cd27-4280-ba31-935a3441e5ad None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.734674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536009b9-cd27-4280-ba31-935a3441e5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.768081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-536009b9-cd27-4280-ba31-935a3441e5ad None None] MFA Rules not processed for user `6bebdfc80ec642e59de13bc670800e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:43.769431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-536009b9-cd27-4280-ba31-935a3441e5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:43.848326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1098/2180] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:43.860985 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea626cee-0f06-4624-bad4-d1e4d885401e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:43.863032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ea626cee-0f06-4624-bad4-d1e4d885401e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:43.929149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea626cee-0f06-4624-bad4-d1e4d885401e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:43.937892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969f688-3a22-468c-9927-fc8223e1477c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:43.938591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969f688-3a22-468c-9927-fc8223e1477c service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:43.939013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969f688-3a22-468c-9927-fc8223e1477c service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:43.939269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969f688-3a22-468c-9927-fc8223e1477c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:43.951417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969f688-3a22-468c-9927-fc8223e1477c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:43.953283 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969f688-3a22-468c-9927-fc8223e1477c service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:43.966036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1083/2181] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:44.369520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536009b9-cd27-4280-ba31-935a3441e5ad None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:44.443981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536009b9-cd27-4280-ba31-935a3441e5ad None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:44.448551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:44.449273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:44.449686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:44.449966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] PATH_INFO: `/v3/users/a341379eb3ef4b31b367cbce41f3d566` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:44.451777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a341379eb3ef4b31b367cbce41f3d566)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:44.453781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:44.527062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a341379eb3ef4b31b367cbce41f3d566'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:44.530803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:44.542687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a341379eb3ef4b31b367cbce41f3d566'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:44.544309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a341379eb3ef4b31b367cbce41f3d566'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:44.545812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a341379eb3ef4b31b367cbce41f3d566'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:44.558648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a341379eb3ef4b31b367cbce41f3d566'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:44.588983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1099/2182] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:44 2022] DELETE /identity/v3/users/a341379eb3ef4b31b367cbce41f3d566 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:44.592877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95de74-add0-4438-a6ab-60fa95268564 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:44.593439 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95de74-add0-4438-a6ab-60fa95268564 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:44.593958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95de74-add0-4438-a6ab-60fa95268564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:44.621114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:44.627851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9a95de74-add0-4438-a6ab-60fa95268564 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:44.628993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9a95de74-add0-4438-a6ab-60fa95268564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:44.766680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1084/2183] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:44.772228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:44.774165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:44.870977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d870a8b-e799-4985-963f-b55da15de1c6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:44.875424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07728681-d372-4bdc-a7bb-8660fc41e1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:44.876222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728681-d372-4bdc-a7bb-8660fc41e1fe service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:44.876680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728681-d372-4bdc-a7bb-8660fc41e1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:44.877185 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728681-d372-4bdc-a7bb-8660fc41e1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:44.895020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07728681-d372-4bdc-a7bb-8660fc41e1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:44.896747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07728681-d372-4bdc-a7bb-8660fc41e1fe service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:44.910561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1100/2184] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:45.289595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95de74-add0-4438-a6ab-60fa95268564 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:45.291372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9a95de74-add0-4438-a6ab-60fa95268564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:45.321965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95de74-add0-4438-a6ab-60fa95268564 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:45.330934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:45.332023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:45.332313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:45.332594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] PATH_INFO: `/v3/projects/b6da3a04825542218ee1a25add941dd1` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:45.333821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6da3a04825542218ee1a25add941dd1)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:45.336432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:45.409461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6da3a04825542218ee1a25add941dd1', 'request_id': 'req-632ce059-828f-4f03-bd04-bb0b5a3ed569'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.417841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6da3a04825542218ee1a25add941dd1', 'request_id': 'req-632ce059-828f-4f03-bd04-bb0b5a3ed569'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.427321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6da3a04825542218ee1a25add941dd1', 'request_id': 'req-632ce059-828f-4f03-bd04-bb0b5a3ed569'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.443218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] The token cache is being invalidate because project b6da3a04825542218ee1a25add941dd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:45.443988 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.444666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.451800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1085/2185] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:45 2022] DELETE /identity/v3/projects/b6da3a04825542218ee1a25add941dd1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:45.452769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07728681-d372-4bdc-a7bb-8660fc41e1fe service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:45.468332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07728681-d372-4bdc-a7bb-8660fc41e1fe service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:45.538729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:45.539479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:45.539658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:45.539953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] PATH_INFO: `/v3/users/6bebdfc80ec642e59de13bc670800e2d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:45.541198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bebdfc80ec642e59de13bc670800e2d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:45.543202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:45.593326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bebdfc80ec642e59de13bc670800e2d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.601292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bebdfc80ec642e59de13bc670800e2d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.601927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bebdfc80ec642e59de13bc670800e2d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.603341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bebdfc80ec642e59de13bc670800e2d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.611878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bebdfc80ec642e59de13bc670800e2d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:45.618927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:45.632624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1101/2186] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:45 2022] DELETE /identity/v3/users/6bebdfc80ec642e59de13bc670800e2d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:45.923502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:45.925042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:45.939142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ce059-828f-4f03-bd04-bb0b5a3ed569 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:46.002570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:46.003204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.003485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.003772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] PATH_INFO: `/v3/projects/9433da7f2dab407c816e7dfccb6ba9dd` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.004929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9433da7f2dab407c816e7dfccb6ba9dd)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:46.006841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:46.042357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:46.053484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.055287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:46.060598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9433da7f2dab407c816e7dfccb6ba9dd', 'request_id': 'req-0721db6a-fd9a-47cc-989d-f88edeb53429'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.066870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9433da7f2dab407c816e7dfccb6ba9dd', 'request_id': 'req-0721db6a-fd9a-47cc-989d-f88edeb53429'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.068901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d24757-48d8-4163-a371-f7523950fcc6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:46.076910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9433da7f2dab407c816e7dfccb6ba9dd', 'request_id': 'req-0721db6a-fd9a-47cc-989d-f88edeb53429'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.088369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] The token cache is being invalidate because project 9433da7f2dab407c816e7dfccb6ba9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:46.088661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.089332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0721db6a-fd9a-47cc-989d-f88edeb53429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.090829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1086/2187] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:45 2022] DELETE /identity/v3/projects/9433da7f2dab407c816e7dfccb6ba9dd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:46.095573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af250c9-ab0b-444f-987b-215a36604b91 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.095894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af250c9-ab0b-444f-987b-215a36604b91 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.096172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af250c9-ab0b-444f-987b-215a36604b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.116521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5af250c9-ab0b-444f-987b-215a36604b91 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:46.117652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5af250c9-ab0b-444f-987b-215a36604b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:46.129974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:46.130646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.130917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.131235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] PATH_INFO: `/v3/users/07de1fb3841b466ca357d48064b3a3ef` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.132553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07de1fb3841b466ca357d48064b3a3ef)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:46.134631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:46.192150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1087/2188] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:46.198760 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07de1fb3841b466ca357d48064b3a3ef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.199898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00968f-1ed7-4230-b4ab-13db3c8f4763 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.200193 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00968f-1ed7-4230-b4ab-13db3c8f4763 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.200602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00968f-1ed7-4230-b4ab-13db3c8f4763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.208747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07de1fb3841b466ca357d48064b3a3ef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.209398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07de1fb3841b466ca357d48064b3a3ef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.210208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07de1fb3841b466ca357d48064b3a3ef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.218931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-64b7c394-270a-4a23-8206-8c087cb6e367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07de1fb3841b466ca357d48064b3a3ef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:46.229909 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9e00968f-1ed7-4230-b4ab-13db3c8f4763 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:46.232083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9e00968f-1ed7-4230-b4ab-13db3c8f4763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:46.241055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1088/2189] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:46.245429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00968f-1ed7-4230-b4ab-13db3c8f4763 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.253269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1102/2190] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:46 2022] DELETE /identity/v3/users/07de1fb3841b466ca357d48064b3a3ef => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:46.257600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba0d67-8332-45a9-8be8-21e05f8a3b53 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.257888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba0d67-8332-45a9-8be8-21e05f8a3b53 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.258166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba0d67-8332-45a9-8be8-21e05f8a3b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.261887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e00968f-1ed7-4230-b4ab-13db3c8f4763 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:46.280233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-45ba0d67-8332-45a9-8be8-21e05f8a3b53 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:46.283004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-45ba0d67-8332-45a9-8be8-21e05f8a3b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:46.342835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:46.343474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.343781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.344067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.345112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:46.346308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:46.348148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:46.354784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1103/2191] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:46.364735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1089/2192] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:22:46.365931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba0d67-8332-45a9-8be8-21e05f8a3b53 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.368684 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.370533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:46.376709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba0d67-8332-45a9-8be8-21e05f8a3b53 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:46.381741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88b5b3-b0a1-40e9-ada4-9e08cc4f4357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:46.382895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88b5b3-b0a1-40e9-ada4-9e08cc4f4357 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.383113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88b5b3-b0a1-40e9-ada4-9e08cc4f4357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.383526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88b5b3-b0a1-40e9-ada4-9e08cc4f4357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.384320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88b5b3-b0a1-40e9-ada4-9e08cc4f4357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:46.386251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88b5b3-b0a1-40e9-ada4-9e08cc4f4357 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:46.393287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:46.428248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:46.434078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1104/2193] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:46 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:46.435742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88b5b3-b0a1-40e9-ada4-9e08cc4f4357 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.448585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5495ae1-a55c-4c40-a640-3f1e4f9230da admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:46.529009 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f39db3-942e-442c-b43b-aab5e1493d90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:46.529686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f39db3-942e-442c-b43b-aab5e1493d90 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.530591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f39db3-942e-442c-b43b-aab5e1493d90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.530870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f39db3-942e-442c-b43b-aab5e1493d90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.542868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f39db3-942e-442c-b43b-aab5e1493d90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:46.545225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f39db3-942e-442c-b43b-aab5e1493d90 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:46.546623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88b5b3-b0a1-40e9-ada4-9e08cc4f4357 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:46.558185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1090/2194] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:22:46.558914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e8852-6f54-4319-b53e-8ad4f5c22ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:46.559542 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e8852-6f54-4319-b53e-8ad4f5c22ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.560445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e8852-6f54-4319-b53e-8ad4f5c22ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.560902 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e8852-6f54-4319-b53e-8ad4f5c22ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.561548 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e8852-6f54-4319-b53e-8ad4f5c22ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:46.563507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e8852-6f54-4319-b53e-8ad4f5c22ae4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:46.615064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1105/2195] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:46 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:46.616495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f39db3-942e-442c-b43b-aab5e1493d90 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.631421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f39db3-942e-442c-b43b-aab5e1493d90 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:46.638748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abc0a0-0aaa-4d84-9de4-a0b8c205af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:46.639539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abc0a0-0aaa-4d84-9de4-a0b8c205af9e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.640737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abc0a0-0aaa-4d84-9de4-a0b8c205af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.640737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abc0a0-0aaa-4d84-9de4-a0b8c205af9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.641528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abc0a0-0aaa-4d84-9de4-a0b8c205af9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:46.643327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abc0a0-0aaa-4d84-9de4-a0b8c205af9e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:46.657423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1091/2196] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:46.661802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e8852-6f54-4319-b53e-8ad4f5c22ae4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.675291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e8852-6f54-4319-b53e-8ad4f5c22ae4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:46.687433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bde688-fc54-405e-ad53-c7f54856dc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:46.688527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bde688-fc54-405e-ad53-c7f54856dc11 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.688837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bde688-fc54-405e-ad53-c7f54856dc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.689208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bde688-fc54-405e-ad53-c7f54856dc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.690199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bde688-fc54-405e-ad53-c7f54856dc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:46.692243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bde688-fc54-405e-ad53-c7f54856dc11 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:46.705793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1106/2197] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:46.710435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abc0a0-0aaa-4d84-9de4-a0b8c205af9e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.721640 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abc0a0-0aaa-4d84-9de4-a0b8c205af9e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:46.727499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0483e-b9b0-42a1-b803-be981948397d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:46.728328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0483e-b9b0-42a1-b803-be981948397d admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.728776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0483e-b9b0-42a1-b803-be981948397d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.729289 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0483e-b9b0-42a1-b803-be981948397d admin admin] PATH_INFO: `/v3/projects/b91069e1a99e47a58a8bda19be64e6c8/users/d115024680ee48c689410349a13d17e1/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.767395 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0483e-b9b0-42a1-b803-be981948397d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b91069e1a99e47a58a8bda19be64e6c8, user_id=d115024680ee48c689410349a13d17e1, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:46.770169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0483e-b9b0-42a1-b803-be981948397d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:46.786972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1092/2198] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:46 2022] PUT /identity/v3/projects/b91069e1a99e47a58a8bda19be64e6c8/users/d115024680ee48c689410349a13d17e1/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:46.798101 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b155c-3f76-409b-bca7-aac5da6ac495 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:46.798462 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b155c-3f76-409b-bca7-aac5da6ac495 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:46.798829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b155c-3f76-409b-bca7-aac5da6ac495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:46.834882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-550b155c-3f76-409b-bca7-aac5da6ac495 None None] MFA Rules not processed for user `d115024680ee48c689410349a13d17e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:46.836305 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-550b155c-3f76-409b-bca7-aac5da6ac495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:46.926019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0483e-b9b0-42a1-b803-be981948397d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.927044 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1107/2199] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:46.927897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e4e0483e-b9b0-42a1-b803-be981948397d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:46.940125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b155c-3f76-409b-bca7-aac5da6ac495 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:46.941932 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-550b155c-3f76-409b-bca7-aac5da6ac495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:47.027017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b155c-3f76-409b-bca7-aac5da6ac495 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:47.027246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0483e-b9b0-42a1-b803-be981948397d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:47.031245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a927c4-79b7-4f5e-af84-fd2cc98cc611 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:47.031883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a927c4-79b7-4f5e-af84-fd2cc98cc611 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:47.032167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a927c4-79b7-4f5e-af84-fd2cc98cc611 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:47.032431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a927c4-79b7-4f5e-af84-fd2cc98cc611 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:47.033167 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:47.034090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:47.034381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:47.034652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] PATH_INFO: `/v3/projects/fc30e1b8d238499baee1a5f94778044d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:47.035865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc30e1b8d238499baee1a5f94778044d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:47.037814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:47.042170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a927c4-79b7-4f5e-af84-fd2cc98cc611 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:47.043804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a927c4-79b7-4f5e-af84-fd2cc98cc611 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:47.050084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:47.059903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1108/2200] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:47.079226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a927c4-79b7-4f5e-af84-fd2cc98cc611 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:47.089917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc30e1b8d238499baee1a5f94778044d', 'request_id': 'req-caf577ce-dd1b-49e4-8d46-893cbe627c3f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.090531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a927c4-79b7-4f5e-af84-fd2cc98cc611 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:47.096861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc30e1b8d238499baee1a5f94778044d', 'request_id': 'req-caf577ce-dd1b-49e4-8d46-893cbe627c3f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.105274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc30e1b8d238499baee1a5f94778044d', 'request_id': 'req-caf577ce-dd1b-49e4-8d46-893cbe627c3f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.112211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:47.117861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] The token cache is being invalidate because project fc30e1b8d238499baee1a5f94778044d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:47.118282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.119466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.122207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1093/2201] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:46 2022] DELETE /identity/v3/projects/fc30e1b8d238499baee1a5f94778044d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:47.126012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:47.127372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:47.158210 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:47.158861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:47.159146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:47.159421 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] PATH_INFO: `/v3/users/d115024680ee48c689410349a13d17e1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:47.160635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d115024680ee48c689410349a13d17e1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:47.162610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:47.202610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf577ce-dd1b-49e4-8d46-893cbe627c3f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:47.207844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:47.208508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:47.208784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:47.209060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] PATH_INFO: `/v3/users/69c021849e0a494181595949aac5b50d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:47.210258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69c021849e0a494181595949aac5b50d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:47.211665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd115024680ee48c689410349a13d17e1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.212254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:47.222882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd115024680ee48c689410349a13d17e1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.223557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd115024680ee48c689410349a13d17e1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.224240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd115024680ee48c689410349a13d17e1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.233217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c42114e3-1674-44a8-9548-7a8b15c97e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd115024680ee48c689410349a13d17e1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.252707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1109/2202] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:47 2022] DELETE /identity/v3/users/d115024680ee48c689410349a13d17e1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:47.257466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a715-2662-4d0d-a1bc-c51c358e7cec None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:47.257835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a715-2662-4d0d-a1bc-c51c358e7cec None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:47.258182 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a715-2662-4d0d-a1bc-c51c358e7cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:47.274847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69c021849e0a494181595949aac5b50d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.275596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69c021849e0a494181595949aac5b50d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.281211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1f10a715-2662-4d0d-a1bc-c51c358e7cec None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:47.282613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1f10a715-2662-4d0d-a1bc-c51c358e7cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:47.285036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69c021849e0a494181595949aac5b50d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.285681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69c021849e0a494181595949aac5b50d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.293579 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69c021849e0a494181595949aac5b50d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.311057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1094/2203] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:47 2022] DELETE /identity/v3/users/69c021849e0a494181595949aac5b50d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:47.347832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1110/2204] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:47.358310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:47.359883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:47.423027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880da9c3-ab5b-4fa3-a445-4871702b9ec2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:47.426756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f049a-1b56-40fc-9d5c-b936d8346b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:47.427483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f049a-1b56-40fc-9d5c-b936d8346b13 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:47.427810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f049a-1b56-40fc-9d5c-b936d8346b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:47.428166 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f049a-1b56-40fc-9d5c-b936d8346b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:47.466905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:47.507408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f049a-1b56-40fc-9d5c-b936d8346b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:47.508917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f049a-1b56-40fc-9d5c-b936d8346b13 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:47.521786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1095/2205] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:47.641668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10a715-2662-4d0d-a1bc-c51c358e7cec None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:47.643291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1f10a715-2662-4d0d-a1bc-c51c358e7cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:47.660150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10a715-2662-4d0d-a1bc-c51c358e7cec None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:47.666509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:47.667231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:47.667584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:47.668002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] PATH_INFO: `/v3/projects/2f5053a0e5c747298d2f64f9f9dacb3e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:47.669797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5053a0e5c747298d2f64f9f9dacb3e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:47.671951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:47.692977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:47.772420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5053a0e5c747298d2f64f9f9dacb3e', 'request_id': 'req-60368347-fc7d-4335-b34b-72489126f69c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.785931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5053a0e5c747298d2f64f9f9dacb3e', 'request_id': 'req-60368347-fc7d-4335-b34b-72489126f69c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.803087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5053a0e5c747298d2f64f9f9dacb3e', 'request_id': 'req-60368347-fc7d-4335-b34b-72489126f69c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.837667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] The token cache is being invalidate because project 2f5053a0e5c747298d2f64f9f9dacb3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:47.837961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.841565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:47.844613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1111/2206] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:47 2022] DELETE /identity/v3/projects/2f5053a0e5c747298d2f64f9f9dacb3e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:47.849816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f049a-1b56-40fc-9d5c-b936d8346b13 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:47.863447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f049a-1b56-40fc-9d5c-b936d8346b13 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:47.947858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:47.947858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:48.012256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.012256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.012256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.012256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] PATH_INFO: `/v3/users/70da8335eed649cf8ff8d064f8fc3f9e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.013940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70da8335eed649cf8ff8d064f8fc3f9e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.013940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.111560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60368347-fc7d-4335-b34b-72489126f69c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.116395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.117126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.117559 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.117888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] PATH_INFO: `/v3/projects/b91069e1a99e47a58a8bda19be64e6c8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.119074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b91069e1a99e47a58a8bda19be64e6c8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.120081 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70da8335eed649cf8ff8d064f8fc3f9e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.121029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70da8335eed649cf8ff8d064f8fc3f9e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.121365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.131123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70da8335eed649cf8ff8d064f8fc3f9e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.132049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70da8335eed649cf8ff8d064f8fc3f9e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.140219 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-8bcee4a6-7f08-4449-a456-2816d0fe12d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70da8335eed649cf8ff8d064f8fc3f9e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.165104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1096/2207] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:47 2022] DELETE /identity/v3/users/70da8335eed649cf8ff8d064f8fc3f9e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:48.186047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b91069e1a99e47a58a8bda19be64e6c8', 'request_id': 'req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.192897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b91069e1a99e47a58a8bda19be64e6c8', 'request_id': 'req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.202230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b91069e1a99e47a58a8bda19be64e6c8', 'request_id': 'req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.217995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] The token cache is being invalidate because project b91069e1a99e47a58a8bda19be64e6c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:48.218291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.218984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-1c7e8466-7cd0-4b76-a886-85a2a4c8edb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.220872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1112/2208] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:47 2022] DELETE /identity/v3/projects/b91069e1a99e47a58a8bda19be64e6c8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:48.226101 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc592-369f-4895-b339-dbfb143d0440 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.226386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc592-369f-4895-b339-dbfb143d0440 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.226659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adc592-369f-4895-b339-dbfb143d0440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.253032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-92adc592-369f-4895-b339-dbfb143d0440 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:48.254319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-92adc592-369f-4895-b339-dbfb143d0440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:48.302993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:48.327951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1097/2209] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:48.337068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2c49-8948-47b2-a5e1-26962b0d0138 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.337353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2c49-8948-47b2-a5e1-26962b0d0138 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.337630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf2c49-8948-47b2-a5e1-26962b0d0138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.360458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-49cf2c49-8948-47b2-a5e1-26962b0d0138 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:48.361686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-49cf2c49-8948-47b2-a5e1-26962b0d0138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:48.369842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1113/2210] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:48.374120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc592-369f-4895-b339-dbfb143d0440 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.384043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adc592-369f-4895-b339-dbfb143d0440 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.388733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97c8f5-45ff-4819-bfa8-165e7de8cb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.389339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c8f5-45ff-4819-bfa8-165e7de8cb9e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.389616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c8f5-45ff-4819-bfa8-165e7de8cb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.389886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97c8f5-45ff-4819-bfa8-165e7de8cb9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.390531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c8f5-45ff-4819-bfa8-165e7de8cb9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:48.390833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c8f5-45ff-4819-bfa8-165e7de8cb9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.392459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97c8f5-45ff-4819-bfa8-165e7de8cb9e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.406937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1098/2211] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:22:48.409212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf2c49-8948-47b2-a5e1-26962b0d0138 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.418841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf2c49-8948-47b2-a5e1-26962b0d0138 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.424337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caaf5c-9109-4b2e-a737-6289b1e5fc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.424945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caaf5c-9109-4b2e-a737-6289b1e5fc2d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.425229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caaf5c-9109-4b2e-a737-6289b1e5fc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.425504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caaf5c-9109-4b2e-a737-6289b1e5fc2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.426139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caaf5c-9109-4b2e-a737-6289b1e5fc2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.428094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caaf5c-9109-4b2e-a737-6289b1e5fc2d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.460484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1114/2212] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:48 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:48.464824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97c8f5-45ff-4819-bfa8-165e7de8cb9e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.475435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caaf5c-9109-4b2e-a737-6289b1e5fc2d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.477958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-87caaf5c-9109-4b2e-a737-6289b1e5fc2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:48.481443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97c8f5-45ff-4819-bfa8-165e7de8cb9e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.483287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:48.550638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caaf5c-9109-4b2e-a737-6289b1e5fc2d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.550903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65213ff2-3fd6-4474-a6a7-92c54da4ae82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.551275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65213ff2-3fd6-4474-a6a7-92c54da4ae82 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.551554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65213ff2-3fd6-4474-a6a7-92c54da4ae82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.551859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65213ff2-3fd6-4474-a6a7-92c54da4ae82 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.552500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65213ff2-3fd6-4474-a6a7-92c54da4ae82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.554414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65213ff2-3fd6-4474-a6a7-92c54da4ae82 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.555424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.556114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.556391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.556664 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] PATH_INFO: `/v3/projects/c20342b2e0c242f5a4668547bf9ca1e6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.557784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20342b2e0c242f5a4668547bf9ca1e6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.559737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.576653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1099/2213] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:48 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:48.580320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65213ff2-3fd6-4474-a6a7-92c54da4ae82 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.591499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65213ff2-3fd6-4474-a6a7-92c54da4ae82 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.596419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894978e3-9975-4dcd-ac21-5fd43b7d8097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.597085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894978e3-9975-4dcd-ac21-5fd43b7d8097 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.597424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894978e3-9975-4dcd-ac21-5fd43b7d8097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.597732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894978e3-9975-4dcd-ac21-5fd43b7d8097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.598410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894978e3-9975-4dcd-ac21-5fd43b7d8097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.600220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894978e3-9975-4dcd-ac21-5fd43b7d8097 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.611080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1100/2214] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:48.614974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894978e3-9975-4dcd-ac21-5fd43b7d8097 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.617655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20342b2e0c242f5a4668547bf9ca1e6', 'request_id': 'req-fa473091-22e5-42be-9bfb-dd7f0a775614'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.624660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894978e3-9975-4dcd-ac21-5fd43b7d8097 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.624842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20342b2e0c242f5a4668547bf9ca1e6', 'request_id': 'req-fa473091-22e5-42be-9bfb-dd7f0a775614'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.632841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20342b2e0c242f5a4668547bf9ca1e6', 'request_id': 'req-fa473091-22e5-42be-9bfb-dd7f0a775614'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.645091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] The token cache is being invalidate because project c20342b2e0c242f5a4668547bf9ca1e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:48.645697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.646296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-fa473091-22e5-42be-9bfb-dd7f0a775614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:48.647576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1115/2215] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:48 2022] DELETE /identity/v3/projects/c20342b2e0c242f5a4668547bf9ca1e6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:48.652682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d385acb-79e2-4b98-9570-3f1f0a3316ee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.652682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d385acb-79e2-4b98-9570-3f1f0a3316ee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.652924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d385acb-79e2-4b98-9570-3f1f0a3316ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.672899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0d385acb-79e2-4b98-9570-3f1f0a3316ee None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:48.674115 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0d385acb-79e2-4b98-9570-3f1f0a3316ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:48.683251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1a133-64e1-41dc-bd76-a5a979af4fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.684096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1a133-64e1-41dc-bd76-a5a979af4fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.684396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1a133-64e1-41dc-bd76-a5a979af4fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.684698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1a133-64e1-41dc-bd76-a5a979af4fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.685334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1a133-64e1-41dc-bd76-a5a979af4fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.686967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1a133-64e1-41dc-bd76-a5a979af4fb3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.697822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1101/2216] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:48.702286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1a133-64e1-41dc-bd76-a5a979af4fb3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.703993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6ea1a133-64e1-41dc-bd76-a5a979af4fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:48.717274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1a133-64e1-41dc-bd76-a5a979af4fb3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.722044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd348da3-a778-4b85-a465-b744cebfc04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.722681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd348da3-a778-4b85-a465-b744cebfc04e admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.722964 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd348da3-a778-4b85-a465-b744cebfc04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.723239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd348da3-a778-4b85-a465-b744cebfc04e admin admin] PATH_INFO: `/v3/projects/65f8fda059f74e06b7f26a7caa24cd02/users/113ad750ad9c4c6e81d6812616e61116/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.734563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1116/2217] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:48.738230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd348da3-a778-4b85-a465-b744cebfc04e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65f8fda059f74e06b7f26a7caa24cd02, user_id=113ad750ad9c4c6e81d6812616e61116, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.740948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd348da3-a778-4b85-a465-b744cebfc04e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.745800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdcd82-0a2b-4a24-80f1-20b409665a4c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.746272 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdcd82-0a2b-4a24-80f1-20b409665a4c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.746587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdcd82-0a2b-4a24-80f1-20b409665a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.755080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1102/2218] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:48 2022] PUT /identity/v3/projects/65f8fda059f74e06b7f26a7caa24cd02/users/113ad750ad9c4c6e81d6812616e61116/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:48.765801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb254b-3266-43a8-b3e7-fe8d552a6fae None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.766553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb254b-3266-43a8-b3e7-fe8d552a6fae None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.766553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb254b-3266-43a8-b3e7-fe8d552a6fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.773036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f3fdcd82-0a2b-4a24-80f1-20b409665a4c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:48.774511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f3fdcd82-0a2b-4a24-80f1-20b409665a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:48.804254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0aeb254b-3266-43a8-b3e7-fe8d552a6fae None None] MFA Rules not processed for user `113ad750ad9c4c6e81d6812616e61116`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:48.805462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0aeb254b-3266-43a8-b3e7-fe8d552a6fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:48.814796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:48.820804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:48.852755 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1117/2219] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:48.856821 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdcd82-0a2b-4a24-80f1-20b409665a4c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.870623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdcd82-0a2b-4a24-80f1-20b409665a4c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.877139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.878005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.878127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.878321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.879147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:48.879794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.881101 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.893948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1103/2220] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:48.894480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1118/2221] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:22:48.898263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb254b-3266-43a8-b3e7-fe8d552a6fae None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.905912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.907877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:48.911282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb254b-3266-43a8-b3e7-fe8d552a6fae None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:48.916906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c2aae-e41c-48cc-a540-dfdd6a76906f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:48.917520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c2aae-e41c-48cc-a540-dfdd6a76906f admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:48.917800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c2aae-e41c-48cc-a540-dfdd6a76906f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:48.918072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c2aae-e41c-48cc-a540-dfdd6a76906f admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:48.918716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c2aae-e41c-48cc-a540-dfdd6a76906f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:48.920708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c2aae-e41c-48cc-a540-dfdd6a76906f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:48.958208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1104/2222] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:48 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:48.962460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c2aae-e41c-48cc-a540-dfdd6a76906f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:48.977414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4452878-2288-4f10-b521-5c945fccad2c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:49.047050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c2aae-e41c-48cc-a540-dfdd6a76906f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:49.047501 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fbc5f-075f-44de-b6fe-f4ce7f157e3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:49.048278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fbc5f-075f-44de-b6fe-f4ce7f157e3b service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:49.048746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fbc5f-075f-44de-b6fe-f4ce7f157e3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:49.049172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fbc5f-075f-44de-b6fe-f4ce7f157e3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:49.052935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35667b0-d73a-43ce-aa28-6ed9229d2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:49.053726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35667b0-d73a-43ce-aa28-6ed9229d2552 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:49.054188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35667b0-d73a-43ce-aa28-6ed9229d2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:49.054558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35667b0-d73a-43ce-aa28-6ed9229d2552 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:49.055210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35667b0-d73a-43ce-aa28-6ed9229d2552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:49.057478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35667b0-d73a-43ce-aa28-6ed9229d2552 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:49.094018 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1105/2223] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:48 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:49.104225 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35667b0-d73a-43ce-aa28-6ed9229d2552 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:49.117608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35667b0-d73a-43ce-aa28-6ed9229d2552 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:49.125164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483d9cd-74c4-4b27-a185-66a527b907a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:49.126093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483d9cd-74c4-4b27-a185-66a527b907a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:49.126375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483d9cd-74c4-4b27-a185-66a527b907a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:49.126650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483d9cd-74c4-4b27-a185-66a527b907a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:49.127356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483d9cd-74c4-4b27-a185-66a527b907a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:49.129287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483d9cd-74c4-4b27-a185-66a527b907a0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:49.135498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fbc5f-075f-44de-b6fe-f4ce7f157e3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:49.137698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fbc5f-075f-44de-b6fe-f4ce7f157e3b service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:49.140950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1106/2224] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:49.144968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483d9cd-74c4-4b27-a185-66a527b907a0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:49.154939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483d9cd-74c4-4b27-a185-66a527b907a0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:49.156296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1119/2225] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:49.160413 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce556c47-8960-4778-9970-22aa66bc7d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:49.160600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce556c47-8960-4778-9970-22aa66bc7d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:49.160947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce556c47-8960-4778-9970-22aa66bc7d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:49.168544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce556c47-8960-4778-9970-22aa66bc7d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:49.170092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce556c47-8960-4778-9970-22aa66bc7d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:49.176176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce556c47-8960-4778-9970-22aa66bc7d92 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:49.198783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1107/2226] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:49.199982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fbc5f-075f-44de-b6fe-f4ce7f157e3b service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:49.215259 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fbc5f-075f-44de-b6fe-f4ce7f157e3b service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:49.228815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50694-ff1a-4de0-a70d-d751265e7e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:49.229659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50694-ff1a-4de0-a70d-d751265e7e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:49.229948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50694-ff1a-4de0-a70d-d751265e7e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:49.230238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df50694-ff1a-4de0-a70d-d751265e7e21 admin admin] PATH_INFO: `/v3/projects/19e7745bc1df472aa25467a3e4f47602/users/85748c19df46423b9eff962e22396c67/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:49.256057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df50694-ff1a-4de0-a70d-d751265e7e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e7745bc1df472aa25467a3e4f47602, user_id=85748c19df46423b9eff962e22396c67, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:49.260491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df50694-ff1a-4de0-a70d-d751265e7e21 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:49.287838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1120/2227] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:49 2022] PUT /identity/v3/projects/19e7745bc1df472aa25467a3e4f47602/users/85748c19df46423b9eff962e22396c67/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:49.290503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b5d23-45af-4423-a8be-850dd4380a31 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:49.290791 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b5d23-45af-4423-a8be-850dd4380a31 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:49.291066 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b5d23-45af-4423-a8be-850dd4380a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:49.326296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c87b5d23-45af-4423-a8be-850dd4380a31 None None] MFA Rules not processed for user `85748c19df46423b9eff962e22396c67`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:49.327920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c87b5d23-45af-4423-a8be-850dd4380a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:49.385311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:49.405955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1108/2228] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:49.422314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50694-ff1a-4de0-a70d-d751265e7e21 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:49.424669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3df50694-ff1a-4de0-a70d-d751265e7e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:49.436196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:49.531677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df50694-ff1a-4de0-a70d-d751265e7e21 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:49.535802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decadf0b-edcc-497c-add7-b8ba61fd6410 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56a94ccca15c462c9e7a5e88b907f420', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:49.536569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decadf0b-edcc-497c-add7-b8ba61fd6410 service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:49.537093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decadf0b-edcc-497c-add7-b8ba61fd6410 service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:49.537428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decadf0b-edcc-497c-add7-b8ba61fd6410 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:49.548253 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decadf0b-edcc-497c-add7-b8ba61fd6410 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:49.553186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decadf0b-edcc-497c-add7-b8ba61fd6410 service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:49.566473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1121/2229] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:56.842484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b5d23-45af-4423-a8be-850dd4380a31 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:56.910404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b5d23-45af-4423-a8be-850dd4380a31 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:56.914984 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:56.915614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:56.915933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:56.916210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] PATH_INFO: `/v3/users/113ad750ad9c4c6e81d6812616e61116` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:56.917368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=113ad750ad9c4c6e81d6812616e61116)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:56.919281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:56.966242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113ad750ad9c4c6e81d6812616e61116'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:56.966894 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113ad750ad9c4c6e81d6812616e61116'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:56.974281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113ad750ad9c4c6e81d6812616e61116'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:56.975081 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113ad750ad9c4c6e81d6812616e61116'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:56.982532 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113ad750ad9c4c6e81d6812616e61116'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:57.001549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1109/2230] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:56 2022] DELETE /identity/v3/users/113ad750ad9c4c6e81d6812616e61116 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:57.006258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c910cd7-736d-45d2-919e-68755914ef0a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:57.006752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c910cd7-736d-45d2-919e-68755914ef0a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:57.006842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c910cd7-736d-45d2-919e-68755914ef0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:57.029116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5c910cd7-736d-45d2-919e-68755914ef0a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:57.030345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c910cd7-736d-45d2-919e-68755914ef0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:57.098407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1122/2231] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:57.108206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:57.109653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:57.168445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680ca60-22df-48dd-8017-66d926645b2e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:57.172108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620cab62-f411-40a6-a622-a1c9947d8d81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:57.172730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cab62-f411-40a6-a622-a1c9947d8d81 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:57.172994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cab62-f411-40a6-a622-a1c9947d8d81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:57.173270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620cab62-f411-40a6-a622-a1c9947d8d81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:57.185240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cab62-f411-40a6-a622-a1c9947d8d81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:57.187197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620cab62-f411-40a6-a622-a1c9947d8d81 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:57.197348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1110/2232] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:57.493263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c910cd7-736d-45d2-919e-68755914ef0a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:57.494963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c910cd7-736d-45d2-919e-68755914ef0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:57.509288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c910cd7-736d-45d2-919e-68755914ef0a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:57.514729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:57.515479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:57.515783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:57.516065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] PATH_INFO: `/v3/projects/65f8fda059f74e06b7f26a7caa24cd02` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:57.517193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65f8fda059f74e06b7f26a7caa24cd02)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:57.519496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:57.576435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65f8fda059f74e06b7f26a7caa24cd02', 'request_id': 'req-7a6fb461-bd22-452b-8c23-8b8757c66375'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:57.583931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65f8fda059f74e06b7f26a7caa24cd02', 'request_id': 'req-7a6fb461-bd22-452b-8c23-8b8757c66375'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:57.591660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65f8fda059f74e06b7f26a7caa24cd02', 'request_id': 'req-7a6fb461-bd22-452b-8c23-8b8757c66375'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:57.607138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] The token cache is being invalidate because project 65f8fda059f74e06b7f26a7caa24cd02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:22:57.607430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:57.608105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7a6fb461-bd22-452b-8c23-8b8757c66375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:57.609586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1123/2233] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:22:57 2022] DELETE /identity/v3/projects/65f8fda059f74e06b7f26a7caa24cd02 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:57.614391 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d303b-4bbb-410a-a012-0b98f591bb20 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:57.614744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d303b-4bbb-410a-a012-0b98f591bb20 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:57.615146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d303b-4bbb-410a-a012-0b98f591bb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:57.634718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b65d303b-4bbb-410a-a012-0b98f591bb20 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:57.636213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b65d303b-4bbb-410a-a012-0b98f591bb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:57.648450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:57.731228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1111/2234] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:57.750840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fe645-29a3-49eb-b5c7-600f7c01e114 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:57.751125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fe645-29a3-49eb-b5c7-600f7c01e114 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:57.751399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85fe645-29a3-49eb-b5c7-600f7c01e114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:57.774360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-b85fe645-29a3-49eb-b5c7-600f7c01e114 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:57.775626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-b85fe645-29a3-49eb-b5c7-600f7c01e114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:57.786381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1124/2235] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:57.789783 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d303b-4bbb-410a-a012-0b98f591bb20 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:57.801837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d303b-4bbb-410a-a012-0b98f591bb20 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:57.806764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccd1a5-545f-4603-bf43-988c9949d50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:57.807376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd1a5-545f-4603-bf43-988c9949d50d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:57.807653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd1a5-545f-4603-bf43-988c9949d50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:57.808001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd1a5-545f-4603-bf43-988c9949d50d admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:57.808685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccd1a5-545f-4603-bf43-988c9949d50d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:22:57.809118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccd1a5-545f-4603-bf43-988c9949d50d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:57.810712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccd1a5-545f-4603-bf43-988c9949d50d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:57.825814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1112/2236] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:22:57.833348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fe645-29a3-49eb-b5c7-600f7c01e114 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:57.843822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85fe645-29a3-49eb-b5c7-600f7c01e114 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:57.849648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4dc8e-a8c2-4a4d-9167-75745de1f03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:57.850266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4dc8e-a8c2-4a4d-9167-75745de1f03d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:57.850551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4dc8e-a8c2-4a4d-9167-75745de1f03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:57.850895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4dc8e-a8c2-4a4d-9167-75745de1f03d admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:57.851681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4dc8e-a8c2-4a4d-9167-75745de1f03d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:57.855194 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4dc8e-a8c2-4a4d-9167-75745de1f03d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:57.860534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:57.896216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1125/2237] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:57 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:57.900896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccd1a5-545f-4603-bf43-988c9949d50d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:57.911503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccd1a5-545f-4603-bf43-988c9949d50d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:57.989534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc36d7e-6b31-49b1-9af3-8c07dd71e89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:57.990569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36d7e-6b31-49b1-9af3-8c07dd71e89a admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:57.991071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36d7e-6b31-49b1-9af3-8c07dd71e89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:57.991523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36d7e-6b31-49b1-9af3-8c07dd71e89a admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:57.992366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc36d7e-6b31-49b1-9af3-8c07dd71e89a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:57.994856 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc36d7e-6b31-49b1-9af3-8c07dd71e89a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.020362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1113/2238] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:57 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:58.024689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4dc8e-a8c2-4a4d-9167-75745de1f03d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.035326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4dc8e-a8c2-4a4d-9167-75745de1f03d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:58.040046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560cde6-6c59-440a-9327-d5c5ac329e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:58.040652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560cde6-6c59-440a-9327-d5c5ac329e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.040971 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560cde6-6c59-440a-9327-d5c5ac329e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.041228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560cde6-6c59-440a-9327-d5c5ac329e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.041863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560cde6-6c59-440a-9327-d5c5ac329e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:58.043445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560cde6-6c59-440a-9327-d5c5ac329e0a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.054245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1126/2239] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:58.058515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc36d7e-6b31-49b1-9af3-8c07dd71e89a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.070451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc36d7e-6b31-49b1-9af3-8c07dd71e89a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:58.075281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ad0ee-e16d-41b3-a8a4-4c0ae50d6ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:58.076016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ad0ee-e16d-41b3-a8a4-4c0ae50d6ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.076365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ad0ee-e16d-41b3-a8a4-4c0ae50d6ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.077369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ad0ee-e16d-41b3-a8a4-4c0ae50d6ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.078093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ad0ee-e16d-41b3-a8a4-4c0ae50d6ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:58.079772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ad0ee-e16d-41b3-a8a4-4c0ae50d6ca1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.090955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1114/2240] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:58.095507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560cde6-6c59-440a-9327-d5c5ac329e0a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.107361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560cde6-6c59-440a-9327-d5c5ac329e0a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:58.112637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab2773-4d77-472e-a17e-84b81c5237c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:58.113294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab2773-4d77-472e-a17e-84b81c5237c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.113572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab2773-4d77-472e-a17e-84b81c5237c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.114011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab2773-4d77-472e-a17e-84b81c5237c9 admin admin] PATH_INFO: `/v3/projects/3b400c240440419591009bf08c86083f/users/04594a39c92549659ba84d388072844f/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.130939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab2773-4d77-472e-a17e-84b81c5237c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b400c240440419591009bf08c86083f, user_id=04594a39c92549659ba84d388072844f, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:58.134191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab2773-4d77-472e-a17e-84b81c5237c9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.156226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1127/2241] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:58 2022] PUT /identity/v3/projects/3b400c240440419591009bf08c86083f/users/04594a39c92549659ba84d388072844f/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:58.168622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a676ae4-1dcf-4b3f-8451-b09654fffb2a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.168921 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a676ae4-1dcf-4b3f-8451-b09654fffb2a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.169745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a676ae4-1dcf-4b3f-8451-b09654fffb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.204568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5a676ae4-1dcf-4b3f-8451-b09654fffb2a None None] MFA Rules not processed for user `04594a39c92549659ba84d388072844f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:58.206721 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5a676ae4-1dcf-4b3f-8451-b09654fffb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:58.289231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1115/2242] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:58.293848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab2773-4d77-472e-a17e-84b81c5237c9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.360330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab2773-4d77-472e-a17e-84b81c5237c9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:58.365137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7b2c4-f52c-434b-a30d-b67043e96a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:58.365742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7b2c4-f52c-434b-a30d-b67043e96a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.366019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7b2c4-f52c-434b-a30d-b67043e96a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.366297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7b2c4-f52c-434b-a30d-b67043e96a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.366924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7b2c4-f52c-434b-a30d-b67043e96a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:58.368856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7b2c4-f52c-434b-a30d-b67043e96a00 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.399935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1128/2243] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:58 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:22:58.403713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a676ae4-1dcf-4b3f-8451-b09654fffb2a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.415493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a676ae4-1dcf-4b3f-8451-b09654fffb2a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:58.419388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:58.481284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dca5a0-63f5-4961-a30e-06ac4b5e9ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:58.481651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca5a0-63f5-4961-a30e-06ac4b5e9ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.481832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca5a0-63f5-4961-a30e-06ac4b5e9ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.482192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca5a0-63f5-4961-a30e-06ac4b5e9ed8 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.483445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dca5a0-63f5-4961-a30e-06ac4b5e9ed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:58.485456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dca5a0-63f5-4961-a30e-06ac4b5e9ed8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.508290 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1116/2244] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:58 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:58.513002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7b2c4-f52c-434b-a30d-b67043e96a00 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.526415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7b2c4-f52c-434b-a30d-b67043e96a00 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:58.531681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb81663-a0d2-4bd6-a717-ee75f3509429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:58.532320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb81663-a0d2-4bd6-a717-ee75f3509429 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.532598 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb81663-a0d2-4bd6-a717-ee75f3509429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.532865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb81663-a0d2-4bd6-a717-ee75f3509429 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.533495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb81663-a0d2-4bd6-a717-ee75f3509429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:58.535087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb81663-a0d2-4bd6-a717-ee75f3509429 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.546915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1129/2245] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:58.551172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dca5a0-63f5-4961-a30e-06ac4b5e9ed8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.574285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dca5a0-63f5-4961-a30e-06ac4b5e9ed8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:58.578980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8a2c8-d8f2-4b34-8774-ba3ecef9f931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:58.579586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8a2c8-d8f2-4b34-8774-ba3ecef9f931 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.579901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8a2c8-d8f2-4b34-8774-ba3ecef9f931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.580167 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8a2c8-d8f2-4b34-8774-ba3ecef9f931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.580800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8a2c8-d8f2-4b34-8774-ba3ecef9f931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:58.582671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8a2c8-d8f2-4b34-8774-ba3ecef9f931 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.596924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1117/2246] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:58.598410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb81663-a0d2-4bd6-a717-ee75f3509429 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.609078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb81663-a0d2-4bd6-a717-ee75f3509429 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:58.614285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4b846-ad6e-443d-b717-b0043882a38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:58.614956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4b846-ad6e-443d-b717-b0043882a38b admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.615238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4b846-ad6e-443d-b717-b0043882a38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.615511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4b846-ad6e-443d-b717-b0043882a38b admin admin] PATH_INFO: `/v3/projects/fe407858de174dfab3c46f19a1c9ddf0/users/6e455e325b1f448893d14b24729a08d2/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.630919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4b846-ad6e-443d-b717-b0043882a38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe407858de174dfab3c46f19a1c9ddf0, user_id=6e455e325b1f448893d14b24729a08d2, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:58.633638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4b846-ad6e-443d-b717-b0043882a38b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.647903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1130/2247] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:58 2022] PUT /identity/v3/projects/fe407858de174dfab3c46f19a1c9ddf0/users/6e455e325b1f448893d14b24729a08d2/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:58.659331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c1f58-b48f-4e48-ac70-305bcf08b721 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.659659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c1f58-b48f-4e48-ac70-305bcf08b721 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.659969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c1f58-b48f-4e48-ac70-305bcf08b721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.690188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9a0c1f58-b48f-4e48-ac70-305bcf08b721 None None] MFA Rules not processed for user `6e455e325b1f448893d14b24729a08d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:58.691401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9a0c1f58-b48f-4e48-ac70-305bcf08b721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:58.776950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1118/2248] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:22:58.782444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4b846-ad6e-443d-b717-b0043882a38b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.869171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4b846-ad6e-443d-b717-b0043882a38b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:58.875283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970b97d-43ad-4b74-a4ba-b74c1094f2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:58.876077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b97d-43ad-4b74-a4ba-b74c1094f2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:58.876861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b97d-43ad-4b74-a4ba-b74c1094f2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:58.876861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b97d-43ad-4b74-a4ba-b74c1094f2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:58.877396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970b97d-43ad-4b74-a4ba-b74c1094f2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:58.879562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970b97d-43ad-4b74-a4ba-b74c1094f2c5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:58.889838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:58.937578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1131/2249] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:22:58 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:22:58.938653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c1f58-b48f-4e48-ac70-305bcf08b721 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:58.966436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c1f58-b48f-4e48-ac70-305bcf08b721 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:59.042430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3608978-0052-4398-b536-3a52c9447b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:59.043339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3608978-0052-4398-b536-3a52c9447b93 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.043922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3608978-0052-4398-b536-3a52c9447b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.044199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3608978-0052-4398-b536-3a52c9447b93 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.045000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3608978-0052-4398-b536-3a52c9447b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:59.048267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3608978-0052-4398-b536-3a52c9447b93 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:59.084432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1119/2250] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:22:58 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:22:59.088212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970b97d-43ad-4b74-a4ba-b74c1094f2c5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:59.101535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970b97d-43ad-4b74-a4ba-b74c1094f2c5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:59.106907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2304ea-8314-467a-a7a2-a08c728b52fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:59.107976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2304ea-8314-467a-a7a2-a08c728b52fc admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.108412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2304ea-8314-467a-a7a2-a08c728b52fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.109181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2304ea-8314-467a-a7a2-a08c728b52fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.110079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2304ea-8314-467a-a7a2-a08c728b52fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:59.112466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3608978-0052-4398-b536-3a52c9447b93 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:59.114258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e3608978-0052-4398-b536-3a52c9447b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:59.116493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2304ea-8314-467a-a7a2-a08c728b52fc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:59.125149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:59.130673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3608978-0052-4398-b536-3a52c9447b93 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:59.131696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1132/2251] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:22:59.135379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2304ea-8314-467a-a7a2-a08c728b52fc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:59.142014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:59.143483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.144512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.145254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] PATH_INFO: `/v3/users/769ebe8d7638498cb70ab1902c2d907d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.147513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=769ebe8d7638498cb70ab1902c2d907d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:59.151335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2304ea-8314-467a-a7a2-a08c728b52fc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:59.151500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:59.157424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8da89-af91-4b56-8dfa-5275cca3759d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:59.158386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8da89-af91-4b56-8dfa-5275cca3759d admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.158586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8da89-af91-4b56-8dfa-5275cca3759d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.158876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8da89-af91-4b56-8dfa-5275cca3759d admin admin] PATH_INFO: `/v3/projects/f916f82aa2d540158ed7529c946661de/users/e50eb880611b4ae39fbaff30034714fc/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.176671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8da89-af91-4b56-8dfa-5275cca3759d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f916f82aa2d540158ed7529c946661de, user_id=e50eb880611b4ae39fbaff30034714fc, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:59.179815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8da89-af91-4b56-8dfa-5275cca3759d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:59.199163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1133/2252] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:59 2022] PUT /identity/v3/projects/f916f82aa2d540158ed7529c946661de/users/e50eb880611b4ae39fbaff30034714fc/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:59.203851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8da89-af91-4b56-8dfa-5275cca3759d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:59.224850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '769ebe8d7638498cb70ab1902c2d907d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:59.226904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '769ebe8d7638498cb70ab1902c2d907d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:59.245504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '769ebe8d7638498cb70ab1902c2d907d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:59.246156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '769ebe8d7638498cb70ab1902c2d907d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:59.263034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f4e030ec-ef82-475f-a62c-ac584b779732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '769ebe8d7638498cb70ab1902c2d907d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:22:59.292937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1120/2253] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:22:59 2022] DELETE /identity/v3/users/769ebe8d7638498cb70ab1902c2d907d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:22:59.298575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8c1e3-51da-4a55-98c8-d9913649d052 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.300085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8c1e3-51da-4a55-98c8-d9913649d052 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.300848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8c1e3-51da-4a55-98c8-d9913649d052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.326713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8da89-af91-4b56-8dfa-5275cca3759d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:59.349067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8ef8c1e3-51da-4a55-98c8-d9913649d052 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:59.352569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8ef8c1e3-51da-4a55-98c8-d9913649d052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:59.408918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233fdb6-ec96-4bb6-8f14-eeb2c1644c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:59.409826 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233fdb6-ec96-4bb6-8f14-eeb2c1644c05 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.410113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233fdb6-ec96-4bb6-8f14-eeb2c1644c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.410391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233fdb6-ec96-4bb6-8f14-eeb2c1644c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.411034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233fdb6-ec96-4bb6-8f14-eeb2c1644c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:59.413247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233fdb6-ec96-4bb6-8f14-eeb2c1644c05 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:59.424322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1134/2254] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:22:59.428621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233fdb6-ec96-4bb6-8f14-eeb2c1644c05 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:59.430361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8233fdb6-ec96-4bb6-8f14-eeb2c1644c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:59.437670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1121/2255] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:59.444874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233fdb6-ec96-4bb6-8f14-eeb2c1644c05 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:59.449594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8c1e3-51da-4a55-98c8-d9913649d052 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:59.449993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c61cb-1af0-4dfb-84d0-cd961ac8b7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:59.450668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c61cb-1af0-4dfb-84d0-cd961ac8b7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.450948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c61cb-1af0-4dfb-84d0-cd961ac8b7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.451130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8ef8c1e3-51da-4a55-98c8-d9913649d052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:59.451245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c61cb-1af0-4dfb-84d0-cd961ac8b7e8 admin admin] PATH_INFO: `/v3/projects/f916f82aa2d540158ed7529c946661de/users/e50eb880611b4ae39fbaff30034714fc/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.453756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c61cb-1af0-4dfb-84d0-cd961ac8b7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f916f82aa2d540158ed7529c946661de, user_id=e50eb880611b4ae39fbaff30034714fc, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:59.456685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c61cb-1af0-4dfb-84d0-cd961ac8b7e8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:59.482307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1135/2256] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:22:59 2022] PUT /identity/v3/projects/f916f82aa2d540158ed7529c946661de/users/e50eb880611b4ae39fbaff30034714fc/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:22:59.493007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf7494-de29-4ea2-aed8-d36315b61822 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.493308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf7494-de29-4ea2-aed8-d36315b61822 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.493762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf7494-de29-4ea2-aed8-d36315b61822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.532093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-64cf7494-de29-4ea2-aed8-d36315b61822 None None] MFA Rules not processed for user `e50eb880611b4ae39fbaff30034714fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:22:59.533267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8c1e3-51da-4a55-98c8-d9913649d052 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:59.533537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64cf7494-de29-4ea2-aed8-d36315b61822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:59.587949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:59.644203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:22:59.648443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d2824-05e8-471c-b5f2-63636445b080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:59.649421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d2824-05e8-471c-b5f2-63636445b080 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.650011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d2824-05e8-471c-b5f2-63636445b080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.650298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d2824-05e8-471c-b5f2-63636445b080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.726327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1136/2257] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:22:59.740191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf7494-de29-4ea2-aed8-d36315b61822 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:22:59.741904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64cf7494-de29-4ea2-aed8-d36315b61822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:59.800241 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d2824-05e8-471c-b5f2-63636445b080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:59.802386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d2824-05e8-471c-b5f2-63636445b080 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:59.824559 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1122/2258] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:22:59.855747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf7494-de29-4ea2-aed8-d36315b61822 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:22:59.864443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8fa79-9fff-4cbf-a7c2-9f63db067a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:22:59.865071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8fa79-9fff-4cbf-a7c2-9f63db067a1b service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:22:59.865354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8fa79-9fff-4cbf-a7c2-9f63db067a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:22:59.865629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8fa79-9fff-4cbf-a7c2-9f63db067a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:22:59.867532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cfa8fa79-9fff-4cbf-a7c2-9f63db067a1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:22:59.959797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8fa79-9fff-4cbf-a7c2-9f63db067a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:22:59.961614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8fa79-9fff-4cbf-a7c2-9f63db067a1b service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:22:59.972640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1137/2259] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:00.470227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d2824-05e8-471c-b5f2-63636445b080 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:00.476357 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b0d2824-05e8-471c-b5f2-63636445b080 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:00.500510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d2824-05e8-471c-b5f2-63636445b080 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:00.506572 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:00.507709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:00.508671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:00.509163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] PATH_INFO: `/v3/projects/2f8142a083184e349a53858cb30d900d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:00.510752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8142a083184e349a53858cb30d900d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:00.512867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:00.572669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8142a083184e349a53858cb30d900d', 'request_id': 'req-1d117416-792b-4620-8319-c47fb5d4a5fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:00.581615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8142a083184e349a53858cb30d900d', 'request_id': 'req-1d117416-792b-4620-8319-c47fb5d4a5fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:00.588970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8142a083184e349a53858cb30d900d', 'request_id': 'req-1d117416-792b-4620-8319-c47fb5d4a5fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:00.600235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] The token cache is being invalidate because project 2f8142a083184e349a53858cb30d900d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:00.600606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:00.601158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1d117416-792b-4620-8319-c47fb5d4a5fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:00.602671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1123/2260] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:00 2022] DELETE /identity/v3/projects/2f8142a083184e349a53858cb30d900d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:00.607583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e09bbc-eac1-46ae-a9d3-ae3e702dc366 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:00.608240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e09bbc-eac1-46ae-a9d3-ae3e702dc366 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:00.608240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e09bbc-eac1-46ae-a9d3-ae3e702dc366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:00.628012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-09e09bbc-eac1-46ae-a9d3-ae3e702dc366 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:00.629207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-09e09bbc-eac1-46ae-a9d3-ae3e702dc366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:00.696467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1138/2261] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:00.705322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e831c1-b511-48c6-8c53-7811dff94d0b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:00.706545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e831c1-b511-48c6-8c53-7811dff94d0b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:00.706545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e831c1-b511-48c6-8c53-7811dff94d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:00.732399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-40e831c1-b511-48c6-8c53-7811dff94d0b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:00.733671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-40e831c1-b511-48c6-8c53-7811dff94d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:00.745840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1124/2262] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:00.749442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e09bbc-eac1-46ae-a9d3-ae3e702dc366 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:00.759804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e09bbc-eac1-46ae-a9d3-ae3e702dc366 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:00.765261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283ce5b-03e3-4f29-992c-073552cc4134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:00.766017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283ce5b-03e3-4f29-992c-073552cc4134 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:00.766159 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283ce5b-03e3-4f29-992c-073552cc4134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:00.766433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283ce5b-03e3-4f29-992c-073552cc4134 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:00.767211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283ce5b-03e3-4f29-992c-073552cc4134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:23:00.770326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283ce5b-03e3-4f29-992c-073552cc4134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:00.772217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283ce5b-03e3-4f29-992c-073552cc4134 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:00.787608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1139/2263] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:23:00.791674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e831c1-b511-48c6-8c53-7811dff94d0b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:00.805505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e831c1-b511-48c6-8c53-7811dff94d0b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:00.810285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70554-94ae-4311-add1-816cf31d419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:00.811465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70554-94ae-4311-add1-816cf31d419d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:00.811810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70554-94ae-4311-add1-816cf31d419d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:00.812084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70554-94ae-4311-add1-816cf31d419d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:00.812722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d70554-94ae-4311-add1-816cf31d419d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:00.814841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d70554-94ae-4311-add1-816cf31d419d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:00.854505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1125/2264] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:23:00 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:23:00.856702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283ce5b-03e3-4f29-992c-073552cc4134 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:00.872207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283ce5b-03e3-4f29-992c-073552cc4134 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:00.878591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:00.948437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8d48-c5fe-4836-859e-1eff85a255eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:00.949056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8d48-c5fe-4836-859e-1eff85a255eb admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:00.949386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8d48-c5fe-4836-859e-1eff85a255eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:00.949680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf8d48-c5fe-4836-859e-1eff85a255eb admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:00.950468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf8d48-c5fe-4836-859e-1eff85a255eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:00.953215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf8d48-c5fe-4836-859e-1eff85a255eb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:00.978825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1140/2265] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:23:00 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:00.980164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70554-94ae-4311-add1-816cf31d419d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:00.990715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70554-94ae-4311-add1-816cf31d419d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:00.995489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa41a8-e294-479c-9b37-7641704d5718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:00.996149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa41a8-e294-479c-9b37-7641704d5718 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:00.996425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa41a8-e294-479c-9b37-7641704d5718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:00.996693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa41a8-e294-479c-9b37-7641704d5718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:00.997334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa41a8-e294-479c-9b37-7641704d5718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:00.999495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa41a8-e294-479c-9b37-7641704d5718 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:01.010467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1126/2266] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:01.014959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8d48-c5fe-4836-859e-1eff85a255eb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:01.025219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf8d48-c5fe-4836-859e-1eff85a255eb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:01.030426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94c82f-4963-41ff-8a69-7b935214f28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:01.031031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94c82f-4963-41ff-8a69-7b935214f28c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:01.031316 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94c82f-4963-41ff-8a69-7b935214f28c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:01.031587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94c82f-4963-41ff-8a69-7b935214f28c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:01.032283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94c82f-4963-41ff-8a69-7b935214f28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:01.033894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94c82f-4963-41ff-8a69-7b935214f28c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:01.045032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1141/2267] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:01.049268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa41a8-e294-479c-9b37-7641704d5718 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:01.059576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa41a8-e294-479c-9b37-7641704d5718 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:01.065612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de944de-df59-41c9-a64c-4a015be9fdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:01.066284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de944de-df59-41c9-a64c-4a015be9fdf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:01.066554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de944de-df59-41c9-a64c-4a015be9fdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:01.066828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de944de-df59-41c9-a64c-4a015be9fdf2 admin admin] PATH_INFO: `/v3/projects/187c155c2d0740ce9f0c9554ac62afa7/users/0a53d3d54ac94c0aad4a0c01dd96c7c9/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:01.083061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de944de-df59-41c9-a64c-4a015be9fdf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=187c155c2d0740ce9f0c9554ac62afa7, user_id=0a53d3d54ac94c0aad4a0c01dd96c7c9, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:01.086276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de944de-df59-41c9-a64c-4a015be9fdf2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:01.101232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1127/2268] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:23:01 2022] PUT /identity/v3/projects/187c155c2d0740ce9f0c9554ac62afa7/users/0a53d3d54ac94c0aad4a0c01dd96c7c9/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:01.113183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef8d67-2e1d-495a-9905-123d02f696ea None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:01.113474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef8d67-2e1d-495a-9905-123d02f696ea None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:01.113751 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef8d67-2e1d-495a-9905-123d02f696ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:01.154218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-0eef8d67-2e1d-495a-9905-123d02f696ea None None] MFA Rules not processed for user `0a53d3d54ac94c0aad4a0c01dd96c7c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:01.156194 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0eef8d67-2e1d-495a-9905-123d02f696ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:01.245761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1142/2269] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:01.250870 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de944de-df59-41c9-a64c-4a015be9fdf2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:01.257491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:01.367728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de944de-df59-41c9-a64c-4a015be9fdf2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:01.372877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52d690-1d3c-4cfe-b40e-2281e02b5cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:01.375740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52d690-1d3c-4cfe-b40e-2281e02b5cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:01.375740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52d690-1d3c-4cfe-b40e-2281e02b5cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:01.375740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52d690-1d3c-4cfe-b40e-2281e02b5cdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:01.375740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52d690-1d3c-4cfe-b40e-2281e02b5cdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:01.379608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52d690-1d3c-4cfe-b40e-2281e02b5cdb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:01.414790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1128/2270] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:23:01 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:01.418607 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef8d67-2e1d-495a-9905-123d02f696ea None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:01.440589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef8d67-2e1d-495a-9905-123d02f696ea None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:01.580556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a7309-dd01-4cbc-bd34-af4d70e44cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:01.582000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a7309-dd01-4cbc-bd34-af4d70e44cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:01.582289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a7309-dd01-4cbc-bd34-af4d70e44cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:01.582564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a7309-dd01-4cbc-bd34-af4d70e44cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:01.583232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a7309-dd01-4cbc-bd34-af4d70e44cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:01.585360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a7309-dd01-4cbc-bd34-af4d70e44cf5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:01.613265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1143/2271] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:23:01 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:23:01.617696 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52d690-1d3c-4cfe-b40e-2281e02b5cdb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:01.633731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52d690-1d3c-4cfe-b40e-2281e02b5cdb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:01.640385 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fb390-82e3-4579-89c3-d67e47bddaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:01.649500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fb390-82e3-4579-89c3-d67e47bddaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:01.649840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fb390-82e3-4579-89c3-d67e47bddaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:01.650117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fb390-82e3-4579-89c3-d67e47bddaf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:01.650904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fb390-82e3-4579-89c3-d67e47bddaf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:01.652931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fb390-82e3-4579-89c3-d67e47bddaf2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:01.671458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1129/2272] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:01.676951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a7309-dd01-4cbc-bd34-af4d70e44cf5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:01.686576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:01.691389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a7309-dd01-4cbc-bd34-af4d70e44cf5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:01.707336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68229b5e-a104-438c-bf41-f3515c09654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:01.708429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68229b5e-a104-438c-bf41-f3515c09654b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:01.709250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68229b5e-a104-438c-bf41-f3515c09654b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:01.709772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68229b5e-a104-438c-bf41-f3515c09654b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:01.710940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68229b5e-a104-438c-bf41-f3515c09654b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:01.713997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68229b5e-a104-438c-bf41-f3515c09654b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:01.740333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1144/2273] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:01.742857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fb390-82e3-4579-89c3-d67e47bddaf2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:01.756686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fb390-82e3-4579-89c3-d67e47bddaf2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:01.768612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7739e1f-d326-44f1-a428-873daba33398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:01.769599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7739e1f-d326-44f1-a428-873daba33398 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:01.770165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7739e1f-d326-44f1-a428-873daba33398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:01.770636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7739e1f-d326-44f1-a428-873daba33398 admin admin] PATH_INFO: `/v3/projects/fbaf9c39b4594eefbeca5404e1033758/users/5043b3794fab442382d329aa12a0f096/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:01.808680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7739e1f-d326-44f1-a428-873daba33398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbaf9c39b4594eefbeca5404e1033758, user_id=5043b3794fab442382d329aa12a0f096, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:01.812352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7739e1f-d326-44f1-a428-873daba33398 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:01.844575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1130/2274] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:23:01 2022] PUT /identity/v3/projects/fbaf9c39b4594eefbeca5404e1033758/users/5043b3794fab442382d329aa12a0f096/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:01.848399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3256d-b0a6-4e6d-bfec-1fe27d61fea0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:01.849113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3256d-b0a6-4e6d-bfec-1fe27d61fea0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:01.849787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3256d-b0a6-4e6d-bfec-1fe27d61fea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:01.896106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a8f3256d-b0a6-4e6d-bfec-1fe27d61fea0 None None] MFA Rules not processed for user `5043b3794fab442382d329aa12a0f096`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:01.897212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a8f3256d-b0a6-4e6d-bfec-1fe27d61fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:01.984845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1145/2275] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:01.989481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7739e1f-d326-44f1-a428-873daba33398 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:02.084007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7739e1f-d326-44f1-a428-873daba33398 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:02.091796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd649c-d41a-4297-a67b-8d7aa5fc8843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:02.093353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd649c-d41a-4297-a67b-8d7aa5fc8843 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.093750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd649c-d41a-4297-a67b-8d7aa5fc8843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.094094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd649c-d41a-4297-a67b-8d7aa5fc8843 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.094860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd649c-d41a-4297-a67b-8d7aa5fc8843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:02.097161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd649c-d41a-4297-a67b-8d7aa5fc8843 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:02.157727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1131/2276] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:23:01 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:02.162256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3256d-b0a6-4e6d-bfec-1fe27d61fea0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:02.181021 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3256d-b0a6-4e6d-bfec-1fe27d61fea0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:02.288581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f4927-e9d2-490d-aeb6-f100fe049bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:02.289338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f4927-e9d2-490d-aeb6-f100fe049bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.289752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f4927-e9d2-490d-aeb6-f100fe049bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.291033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f4927-e9d2-490d-aeb6-f100fe049bf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.291791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f4927-e9d2-490d-aeb6-f100fe049bf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:02.294104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f4927-e9d2-490d-aeb6-f100fe049bf6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:02.324058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1146/2277] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:23:02 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:02.328789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd649c-d41a-4297-a67b-8d7aa5fc8843 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:02.341638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd649c-d41a-4297-a67b-8d7aa5fc8843 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:02.352661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc163-fa51-4192-ba5a-0f8b5ca7e356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:02.354411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc163-fa51-4192-ba5a-0f8b5ca7e356 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.354685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc163-fa51-4192-ba5a-0f8b5ca7e356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.354961 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cc163-fa51-4192-ba5a-0f8b5ca7e356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.355608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc163-fa51-4192-ba5a-0f8b5ca7e356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:02.357401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cc163-fa51-4192-ba5a-0f8b5ca7e356 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:02.370207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1132/2278] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:23:02.375139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f4927-e9d2-490d-aeb6-f100fe049bf6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:02.388624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f4927-e9d2-490d-aeb6-f100fe049bf6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:02.396136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c58b46-27b6-4fd7-9154-9ec108e59727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:02.396799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c58b46-27b6-4fd7-9154-9ec108e59727 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.397080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c58b46-27b6-4fd7-9154-9ec108e59727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.397474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c58b46-27b6-4fd7-9154-9ec108e59727 admin admin] PATH_INFO: `/v3/projects/d883083f8a3c4f07a1208a4034c212fc/users/216d6257a13349cabc7fd5fd4e8af192/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.413828 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc163-fa51-4192-ba5a-0f8b5ca7e356 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:02.417125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-971cc163-fa51-4192-ba5a-0f8b5ca7e356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:02.421032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c58b46-27b6-4fd7-9154-9ec108e59727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d883083f8a3c4f07a1208a4034c212fc, user_id=216d6257a13349cabc7fd5fd4e8af192, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:02.424032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c58b46-27b6-4fd7-9154-9ec108e59727 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:02.435661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cc163-fa51-4192-ba5a-0f8b5ca7e356 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:02.444236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1147/2279] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:23:02 2022] PUT /identity/v3/projects/d883083f8a3c4f07a1208a4034c212fc/users/216d6257a13349cabc7fd5fd4e8af192/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:23:02.448646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c58b46-27b6-4fd7-9154-9ec108e59727 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:02.474178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:02.520772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:02.531414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:02.532462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.532970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.533423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] PATH_INFO: `/v3/users/2ab0433dd3864867b03c5dd6407aa388` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.536451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab0433dd3864867b03c5dd6407aa388)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:02.538745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:02.541470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c58b46-27b6-4fd7-9154-9ec108e59727 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:02.549832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cd21e-3014-4891-9efd-d39b43d06f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:02.550479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cd21e-3014-4891-9efd-d39b43d06f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.551296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cd21e-3014-4891-9efd-d39b43d06f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.551632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cd21e-3014-4891-9efd-d39b43d06f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.552404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cd21e-3014-4891-9efd-d39b43d06f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:02.555047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cd21e-3014-4891-9efd-d39b43d06f7f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:02.583584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1148/2280] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:23:02.593988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cd21e-3014-4891-9efd-d39b43d06f7f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:02.620025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cd21e-3014-4891-9efd-d39b43d06f7f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:02.633334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c0e11-c474-415d-bd93-6089ad9e0194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:02.634001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c0e11-c474-415d-bd93-6089ad9e0194 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.634281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c0e11-c474-415d-bd93-6089ad9e0194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.635735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c0e11-c474-415d-bd93-6089ad9e0194 admin admin] PATH_INFO: `/v3/projects/d883083f8a3c4f07a1208a4034c212fc/users/216d6257a13349cabc7fd5fd4e8af192/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.638161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c0e11-c474-415d-bd93-6089ad9e0194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d883083f8a3c4f07a1208a4034c212fc, user_id=216d6257a13349cabc7fd5fd4e8af192, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:02.641833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c0e11-c474-415d-bd93-6089ad9e0194 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:02.643970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab0433dd3864867b03c5dd6407aa388'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:02.645048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab0433dd3864867b03c5dd6407aa388'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:02.659146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab0433dd3864867b03c5dd6407aa388'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:02.663827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1149/2281] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:23:02 2022] PUT /identity/v3/projects/d883083f8a3c4f07a1208a4034c212fc/users/216d6257a13349cabc7fd5fd4e8af192/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:02.668090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab0433dd3864867b03c5dd6407aa388'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:02.674854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59301c3c-e0c2-4d37-8001-97d3b67101dc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.676446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59301c3c-e0c2-4d37-8001-97d3b67101dc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.677163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59301c3c-e0c2-4d37-8001-97d3b67101dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.683898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab0433dd3864867b03c5dd6407aa388'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:02.718263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-59301c3c-e0c2-4d37-8001-97d3b67101dc None None] MFA Rules not processed for user `216d6257a13349cabc7fd5fd4e8af192`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:02.719593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-59301c3c-e0c2-4d37-8001-97d3b67101dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:02.733638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1133/2282] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:02 2022] DELETE /identity/v3/users/2ab0433dd3864867b03c5dd6407aa388 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:02.793525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:02.795139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:02.867804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1150/2283] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:02.877877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59301c3c-e0c2-4d37-8001-97d3b67101dc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:02.880340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-59301c3c-e0c2-4d37-8001-97d3b67101dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:02.911589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff15d24-ef54-4f18-828d-e8f10892f6e8 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:02.917187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:02.917812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.918144 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.918468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.921444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:02.987310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59301c3c-e0c2-4d37-8001-97d3b67101dc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:02.991564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ea93e-5627-44a4-9ea5-12ad746ddf99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:02.992466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ea93e-5627-44a4-9ea5-12ad746ddf99 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:02.992907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ea93e-5627-44a4-9ea5-12ad746ddf99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:02.993353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ea93e-5627-44a4-9ea5-12ad746ddf99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:02.995730 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-148ea93e-5627-44a4-9ea5-12ad746ddf99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:03.015440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.017399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.041584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:03.051065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1134/2284] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:23:03.081339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:03.083053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:03.102638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ea93e-5627-44a4-9ea5-12ad746ddf99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.104288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ea93e-5627-44a4-9ea5-12ad746ddf99 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.117708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1151/2285] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:03.178332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7044303-2a80-4e2e-bf05-9342888c21f1 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:03.184420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:03.185341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.185738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.186761 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] PATH_INFO: `/v3/projects/74d5d713723f4ea5a8d1e72627b80c47` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.188005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d5d713723f4ea5a8d1e72627b80c47)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.190006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.258650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d5d713723f4ea5a8d1e72627b80c47', 'request_id': 'req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:03.267875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d5d713723f4ea5a8d1e72627b80c47', 'request_id': 'req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:03.277840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d5d713723f4ea5a8d1e72627b80c47', 'request_id': 'req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:03.296846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] The token cache is being invalidate because project 74d5d713723f4ea5a8d1e72627b80c47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:03.297436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:03.298493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-e22ec8b2-fcf9-481c-a8ed-e002bd06b643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:03.300053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1135/2286] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:03 2022] DELETE /identity/v3/projects/74d5d713723f4ea5a8d1e72627b80c47 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:23:03.305539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df185946-314c-4d15-b36d-dcf5737b68d8 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.305938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df185946-314c-4d15-b36d-dcf5737b68d8 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.306320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df185946-314c-4d15-b36d-dcf5737b68d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.336928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-df185946-314c-4d15-b36d-dcf5737b68d8 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:03.338529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-df185946-314c-4d15-b36d-dcf5737b68d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:03.344993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:03.452048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1152/2287] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:23:03.461109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdae1a-844b-444f-9c9d-096d0969f6ad None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.461989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdae1a-844b-444f-9c9d-096d0969f6ad None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.462479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdae1a-844b-444f-9c9d-096d0969f6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.494740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-03fdae1a-844b-444f-9c9d-096d0969f6ad None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:03.496927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-03fdae1a-844b-444f-9c9d-096d0969f6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:03.507407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1136/2288] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:03.512464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df185946-314c-4d15-b36d-dcf5737b68d8 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:03.529246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df185946-314c-4d15-b36d-dcf5737b68d8 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:03.551546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78c551-6acf-4434-8422-87bd1416a9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:03.557854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78c551-6acf-4434-8422-87bd1416a9dc admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.558246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78c551-6acf-4434-8422-87bd1416a9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.558519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78c551-6acf-4434-8422-87bd1416a9dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.559274 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78c551-6acf-4434-8422-87bd1416a9dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:23:03.559589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78c551-6acf-4434-8422-87bd1416a9dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.568613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78c551-6acf-4434-8422-87bd1416a9dc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.595802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1153/2289] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:23:03.600520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdae1a-844b-444f-9c9d-096d0969f6ad None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:03.610063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdae1a-844b-444f-9c9d-096d0969f6ad None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:03.615678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f5547-3267-4c74-9378-1f593f1db4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:03.616589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f5547-3267-4c74-9378-1f593f1db4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.616589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f5547-3267-4c74-9378-1f593f1db4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.616960 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f5547-3267-4c74-9378-1f593f1db4b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.617642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f5547-3267-4c74-9378-1f593f1db4b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.619570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f5547-3267-4c74-9378-1f593f1db4b0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.654424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1137/2290] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:23:03 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:03.658639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78c551-6acf-4434-8422-87bd1416a9dc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:03.675562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78c551-6acf-4434-8422-87bd1416a9dc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:03.735809 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda292c8-36bc-43c1-9ca5-10f7bc24a86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:03.736432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda292c8-36bc-43c1-9ca5-10f7bc24a86c admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.736714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda292c8-36bc-43c1-9ca5-10f7bc24a86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.736987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda292c8-36bc-43c1-9ca5-10f7bc24a86c admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.737622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda292c8-36bc-43c1-9ca5-10f7bc24a86c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.739570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda292c8-36bc-43c1-9ca5-10f7bc24a86c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.770524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1154/2291] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:23:03 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:03.774525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f5547-3267-4c74-9378-1f593f1db4b0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:03.785958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f5547-3267-4c74-9378-1f593f1db4b0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:03.790941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f31537-094b-4ea4-b09a-313bc5559f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:03.791613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f31537-094b-4ea4-b09a-313bc5559f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.791916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f31537-094b-4ea4-b09a-313bc5559f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.792184 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f31537-094b-4ea4-b09a-313bc5559f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.792815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f31537-094b-4ea4-b09a-313bc5559f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.794396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f31537-094b-4ea4-b09a-313bc5559f89 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.805574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1138/2292] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:03.808255 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda292c8-36bc-43c1-9ca5-10f7bc24a86c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:03.809742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f31537-094b-4ea4-b09a-313bc5559f89 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:03.810311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cda292c8-36bc-43c1-9ca5-10f7bc24a86c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:03.821448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f31537-094b-4ea4-b09a-313bc5559f89 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:03.826210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0303d4-8fb1-41b1-a07f-7bfbc2c708ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:03.826854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0303d4-8fb1-41b1-a07f-7bfbc2c708ca admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.827135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0303d4-8fb1-41b1-a07f-7bfbc2c708ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.827404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0303d4-8fb1-41b1-a07f-7bfbc2c708ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.828091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0303d4-8fb1-41b1-a07f-7bfbc2c708ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.829671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0303d4-8fb1-41b1-a07f-7bfbc2c708ca admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.839850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1139/2293] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:23:03.844556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0303d4-8fb1-41b1-a07f-7bfbc2c708ca admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:03.855347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0303d4-8fb1-41b1-a07f-7bfbc2c708ca admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:03.863662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe8925-651b-4d42-b41a-472623c65c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:03.864363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe8925-651b-4d42-b41a-472623c65c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.864642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe8925-651b-4d42-b41a-472623c65c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.864911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe8925-651b-4d42-b41a-472623c65c84 admin admin] PATH_INFO: `/v3/projects/d3bd02ae79fd4edb9886b1b4ef3d761a/users/0fb9105e474a4a29904722b1e33d0155/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.880371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda292c8-36bc-43c1-9ca5-10f7bc24a86c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:03.881527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe8925-651b-4d42-b41a-472623c65c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3bd02ae79fd4edb9886b1b4ef3d761a, user_id=0fb9105e474a4a29904722b1e33d0155, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.884270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe8925-651b-4d42-b41a-472623c65c84 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.885580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:03.886201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.886543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.886745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.888750 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:03.899469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1140/2294] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:23:03 2022] PUT /identity/v3/projects/d3bd02ae79fd4edb9886b1b4ef3d761a/users/0fb9105e474a4a29904722b1e33d0155/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:23:03.905134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29329039-f226-42ac-b740-10822c4f41ac None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:03.905528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29329039-f226-42ac-b740-10822c4f41ac None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:03.905876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29329039-f226-42ac-b740-10822c4f41ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:03.928347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-29329039-f226-42ac-b740-10822c4f41ac None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:03.929589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-29329039-f226-42ac-b740-10822c4f41ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:03.939639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:03.944595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:03.970664 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:03.972376 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:03.984373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1155/2295] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:03.999605 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1141/2296] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:23:04.009599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:04.010862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:04.083267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd7c5a-5515-4bd4-b2bb-4cdd028855e0 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:04.111390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7ea9-a659-492e-b179-27d5a8b7907f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:04.111884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7ea9-a659-492e-b179-27d5a8b7907f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:04.112553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b7ea9-a659-492e-b179-27d5a8b7907f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:04.134991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2f0b7ea9-a659-492e-b179-27d5a8b7907f None None] MFA Rules not processed for user `1f8e995175ec4c6583e1ad0c7fb902ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:04.136292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2f0b7ea9-a659-492e-b179-27d5a8b7907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:04.151320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81162878-6f5d-4733-bd08-f80f79ddcb02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:04.151539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81162878-6f5d-4733-bd08-f80f79ddcb02 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:04.151800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81162878-6f5d-4733-bd08-f80f79ddcb02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:04.152009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81162878-6f5d-4733-bd08-f80f79ddcb02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:04.162128 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81162878-6f5d-4733-bd08-f80f79ddcb02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:04.163572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81162878-6f5d-4733-bd08-f80f79ddcb02 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:04.175043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1156/2297] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:04.195186 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1142/2298] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:23:05.942880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81162878-6f5d-4733-bd08-f80f79ddcb02 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:06.013888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81162878-6f5d-4733-bd08-f80f79ddcb02 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:06.020151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:06.021270 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:06.021650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:06.022025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:06.024888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:06.097851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:06.110464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:06.112119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:06.123411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1157/2299] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:07.035060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b7ea9-a659-492e-b179-27d5a8b7907f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:07.045749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b7ea9-a659-492e-b179-27d5a8b7907f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:07.049742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834a455-915f-4f7d-94b9-c0dac8aff982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:07.050823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834a455-915f-4f7d-94b9-c0dac8aff982 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:07.051207 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834a455-915f-4f7d-94b9-c0dac8aff982 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:07.051726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834a455-915f-4f7d-94b9-c0dac8aff982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:07.056811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7834a455-915f-4f7d-94b9-c0dac8aff982 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:07.149920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834a455-915f-4f7d-94b9-c0dac8aff982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:07.152272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834a455-915f-4f7d-94b9-c0dac8aff982 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:07.173589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1143/2300] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:09.338453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:09.342453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:09.370689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4c7a1-aee7-4ac3-84fa-196992ce9868 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:09.377051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:09.378347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:09.378812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:09.379401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] PATH_INFO: `/v3/users/04594a39c92549659ba84d388072844f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:09.384328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04594a39c92549659ba84d388072844f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:09.387778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:09.466477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04594a39c92549659ba84d388072844f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:09.497619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04594a39c92549659ba84d388072844f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:09.498835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04594a39c92549659ba84d388072844f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:09.499476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04594a39c92549659ba84d388072844f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:09.509610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04594a39c92549659ba84d388072844f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:09.540432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1158/2301] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:09 2022] DELETE /identity/v3/users/04594a39c92549659ba84d388072844f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:09.557641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6798-c154-487a-ba66-8ca902f9184a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:09.558140 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6798-c154-487a-ba66-8ca902f9184a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:09.558220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6798-c154-487a-ba66-8ca902f9184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:09.589012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f8ef6798-c154-487a-ba66-8ca902f9184a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:09.591032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f8ef6798-c154-487a-ba66-8ca902f9184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:09.661834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:09.678244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1144/2302] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:09.693869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:09.700405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:09.814619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a7875-9bcf-4e39-b52c-01e3380455b4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:09.819541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0da4e-c340-49cd-8006-2187f029426f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:09.820663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0da4e-c340-49cd-8006-2187f029426f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:09.821043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0da4e-c340-49cd-8006-2187f029426f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:09.821469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0da4e-c340-49cd-8006-2187f029426f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:09.854090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0da4e-c340-49cd-8006-2187f029426f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:09.856666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0da4e-c340-49cd-8006-2187f029426f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:09.869053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1159/2303] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:10.343093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299e5d1-ec10-4d76-9dc9-85f5e6a66fc3 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:10.343780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299e5d1-ec10-4d76-9dc9-85f5e6a66fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:10.344286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299e5d1-ec10-4d76-9dc9-85f5e6a66fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:10.393423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6299e5d1-ec10-4d76-9dc9-85f5e6a66fc3 None None] MFA Rules not processed for user `0fb9105e474a4a29904722b1e33d0155`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:10.395157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6299e5d1-ec10-4d76-9dc9-85f5e6a66fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:10.524691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1145/2304] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:10.535166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e1444-ce9f-4f04-b0ce-8a6444c450a1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:10.535747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e1444-ce9f-4f04-b0ce-8a6444c450a1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:10.535747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e1444-ce9f-4f04-b0ce-8a6444c450a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:10.559806 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-116e1444-ce9f-4f04-b0ce-8a6444c450a1 None None] MFA Rules not processed for user `0fb9105e474a4a29904722b1e33d0155`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:10.560942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-116e1444-ce9f-4f04-b0ce-8a6444c450a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:10.571422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1160/2305] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:10.589080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299e5d1-ec10-4d76-9dc9-85f5e6a66fc3 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:10.594442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6299e5d1-ec10-4d76-9dc9-85f5e6a66fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:10.625109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e1444-ce9f-4f04-b0ce-8a6444c450a1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:10.626934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-116e1444-ce9f-4f04-b0ce-8a6444c450a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:10.655305 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e1444-ce9f-4f04-b0ce-8a6444c450a1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:10.665057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:10.665672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:10.665956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:10.666297 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] PATH_INFO: `/v3/projects/3b400c240440419591009bf08c86083f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:10.667535 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b400c240440419591009bf08c86083f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:10.676142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:10.748695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299e5d1-ec10-4d76-9dc9-85f5e6a66fc3 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:10.768844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b400c240440419591009bf08c86083f', 'request_id': 'req-ef477b84-cce4-4bb6-b244-2b3c3f558707'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:10.776143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b400c240440419591009bf08c86083f', 'request_id': 'req-ef477b84-cce4-4bb6-b244-2b3c3f558707'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:10.778032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:10.802046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b400c240440419591009bf08c86083f', 'request_id': 'req-ef477b84-cce4-4bb6-b244-2b3c3f558707'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:10.815719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] The token cache is being invalidate because project 3b400c240440419591009bf08c86083f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:10.816053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:10.816854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:10.818686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1161/2306] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:10 2022] DELETE /identity/v3/projects/3b400c240440419591009bf08c86083f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:23:10.822437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:10.824281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:10.852570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a796fc-02df-4e90-9437-662c6c5c1662 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:10.853203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a796fc-02df-4e90-9437-662c6c5c1662 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:10.853527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a796fc-02df-4e90-9437-662c6c5c1662 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:10.853816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a796fc-02df-4e90-9437-662c6c5c1662 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:10.856529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-98a796fc-02df-4e90-9437-662c6c5c1662 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:10.881355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:10.915146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef477b84-cce4-4bb6-b244-2b3c3f558707 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:10.922059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:10.922718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:10.923004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:10.932354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] PATH_INFO: `/v3/users/6e455e325b1f448893d14b24729a08d2` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:10.936061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e455e325b1f448893d14b24729a08d2)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:10.938325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:10.949163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a796fc-02df-4e90-9437-662c6c5c1662 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:10.952280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a796fc-02df-4e90-9437-662c6c5c1662 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:10.973720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1146/2307] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:11.030981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e455e325b1f448893d14b24729a08d2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.043338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e455e325b1f448893d14b24729a08d2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.044139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e455e325b1f448893d14b24729a08d2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.045242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e455e325b1f448893d14b24729a08d2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.058047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e455e325b1f448893d14b24729a08d2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.097627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1162/2308] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:10 2022] DELETE /identity/v3/users/6e455e325b1f448893d14b24729a08d2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:23:11.488639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a796fc-02df-4e90-9437-662c6c5c1662 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:11.510670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a796fc-02df-4e90-9437-662c6c5c1662 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:11.631156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:11.631698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:11.632183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:11.633455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] PATH_INFO: `/v3/projects/fe407858de174dfab3c46f19a1c9ddf0` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:11.635011 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe407858de174dfab3c46f19a1c9ddf0)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:11.638140 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:11.726785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe407858de174dfab3c46f19a1c9ddf0', 'request_id': 'req-bb5dafae-1060-4586-8201-b6b7962082bc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.735446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe407858de174dfab3c46f19a1c9ddf0', 'request_id': 'req-bb5dafae-1060-4586-8201-b6b7962082bc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.761431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe407858de174dfab3c46f19a1c9ddf0', 'request_id': 'req-bb5dafae-1060-4586-8201-b6b7962082bc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.774878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] The token cache is being invalidate because project fe407858de174dfab3c46f19a1c9ddf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:11.775165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.775871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bb5dafae-1060-4586-8201-b6b7962082bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:11.777525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1147/2309] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:11 2022] DELETE /identity/v3/projects/fe407858de174dfab3c46f19a1c9ddf0 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:11.781675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:11.783200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:11.799257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed2486-88ce-4a1f-bc4f-4105a38c9c0a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:11.908893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:11.909762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:11.910048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:11.912125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] PATH_INFO: `/v3/users/e50eb880611b4ae39fbaff30034714fc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:11.914920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e50eb880611b4ae39fbaff30034714fc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:11.921588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:11.930763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:11.991352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ab222-c4d1-4bda-bad0-99440a3afae5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:11.991633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ab222-c4d1-4bda-bad0-99440a3afae5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:11.992030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ab222-c4d1-4bda-bad0-99440a3afae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:12.046810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b25ab222-c4d1-4bda-bad0-99440a3afae5 None None] MFA Rules not processed for user `0fb9105e474a4a29904722b1e33d0155`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:12.048106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b25ab222-c4d1-4bda-bad0-99440a3afae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:12.076037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e50eb880611b4ae39fbaff30034714fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.091417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e50eb880611b4ae39fbaff30034714fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.092840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e50eb880611b4ae39fbaff30034714fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.093471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e50eb880611b4ae39fbaff30034714fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.106588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e50eb880611b4ae39fbaff30034714fc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.124996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:12.131796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1163/2310] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:11 2022] DELETE /identity/v3/users/e50eb880611b4ae39fbaff30034714fc => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:12.164978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1148/2311] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:23:12.632655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:12.634467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:12.648699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ab222-c4d1-4bda-bad0-99440a3afae5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:12.650571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b25ab222-c4d1-4bda-bad0-99440a3afae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:12.693525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ab222-c4d1-4bda-bad0-99440a3afae5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:12.799966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:12.801105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ae45c-591e-43fc-b12c-1c310bbc21e6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:12.801662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:12.802162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:12.807834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] PATH_INFO: `/v3/users/0a53d3d54ac94c0aad4a0c01dd96c7c9` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:12.810042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a53d3d54ac94c0aad4a0c01dd96c7c9)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:12.813459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:12.815781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:12.816503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:12.816807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:12.821694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] PATH_INFO: `/v3/projects/f916f82aa2d540158ed7529c946661de` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:12.823326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f916f82aa2d540158ed7529c946661de)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:12.826831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:12.936646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a53d3d54ac94c0aad4a0c01dd96c7c9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.938412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a53d3d54ac94c0aad4a0c01dd96c7c9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.945609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f916f82aa2d540158ed7529c946661de', 'request_id': 'req-37240a44-1956-4997-acc4-bb316d6949e2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.954127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a53d3d54ac94c0aad4a0c01dd96c7c9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.954737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a53d3d54ac94c0aad4a0c01dd96c7c9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.963352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f916f82aa2d540158ed7529c946661de', 'request_id': 'req-37240a44-1956-4997-acc4-bb316d6949e2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.983706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f916f82aa2d540158ed7529c946661de', 'request_id': 'req-37240a44-1956-4997-acc4-bb316d6949e2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:12.999567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a53d3d54ac94c0aad4a0c01dd96c7c9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:13.006943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] The token cache is being invalidate because project f916f82aa2d540158ed7529c946661de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:13.007339 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:13.008304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-37240a44-1956-4997-acc4-bb316d6949e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:13.011277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1164/2312] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:12 2022] DELETE /identity/v3/projects/f916f82aa2d540158ed7529c946661de => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:23:13.072267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1149/2313] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:12 2022] DELETE /identity/v3/users/0a53d3d54ac94c0aad4a0c01dd96c7c9 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:23:13.085560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b4413-a6a8-4c81-ac9a-6760f5d0e4b0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:13.085560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b4413-a6a8-4c81-ac9a-6760f5d0e4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:13.085560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b4413-a6a8-4c81-ac9a-6760f5d0e4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:13.193742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-898b4413-a6a8-4c81-ac9a-6760f5d0e4b0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:13.209379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-898b4413-a6a8-4c81-ac9a-6760f5d0e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:13.241686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:13.369654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1165/2314] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:13.380381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:13.382514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:13.522422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496f1c3-da45-4e9d-9cf6-324c9a35606d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:13.535280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4709306-79a2-4017-8eb6-6075b64cb9fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:13.536198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4709306-79a2-4017-8eb6-6075b64cb9fc service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:13.536576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4709306-79a2-4017-8eb6-6075b64cb9fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:13.548410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4709306-79a2-4017-8eb6-6075b64cb9fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:13.554434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4709306-79a2-4017-8eb6-6075b64cb9fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:13.554434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4709306-79a2-4017-8eb6-6075b64cb9fc service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:13.568249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1150/2315] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:14.146219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b4413-a6a8-4c81-ac9a-6760f5d0e4b0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:14.147945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-898b4413-a6a8-4c81-ac9a-6760f5d0e4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:14.172334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b4413-a6a8-4c81-ac9a-6760f5d0e4b0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:14.177314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:14.178136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:14.178422 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:14.178698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] PATH_INFO: `/v3/projects/187c155c2d0740ce9f0c9554ac62afa7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:14.180132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=187c155c2d0740ce9f0c9554ac62afa7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:14.182677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:14.333056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '187c155c2d0740ce9f0c9554ac62afa7', 'request_id': 'req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.345112 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '187c155c2d0740ce9f0c9554ac62afa7', 'request_id': 'req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.357396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '187c155c2d0740ce9f0c9554ac62afa7', 'request_id': 'req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.371842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] The token cache is being invalidate because project 187c155c2d0740ce9f0c9554ac62afa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:14.372193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.373512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.375395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1166/2316] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:14 2022] DELETE /identity/v3/projects/187c155c2d0740ce9f0c9554ac62afa7 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:14.379348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4709306-79a2-4017-8eb6-6075b64cb9fc service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:14.391971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4709306-79a2-4017-8eb6-6075b64cb9fc service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:14.407445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:14.532328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:14.533004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:14.533291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:14.533570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] PATH_INFO: `/v3/users/5043b3794fab442382d329aa12a0f096` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:14.540896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5043b3794fab442382d329aa12a0f096)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:14.543034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:14.633123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5043b3794fab442382d329aa12a0f096'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.633873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5043b3794fab442382d329aa12a0f096'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.645314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5043b3794fab442382d329aa12a0f096'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.646127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5043b3794fab442382d329aa12a0f096'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.655027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5043b3794fab442382d329aa12a0f096'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:14.680735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1151/2317] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:14 2022] DELETE /identity/v3/users/5043b3794fab442382d329aa12a0f096 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:15.099479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:15.101561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:15.119033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac1a35-0de5-47cd-a0cf-8fafee1df897 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:15.216198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:15.217969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:15.218486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:15.218972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] PATH_INFO: `/v3/projects/fbaf9c39b4594eefbeca5404e1033758` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:15.221726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbaf9c39b4594eefbeca5404e1033758)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:15.223976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:15.282561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:15.300086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbaf9c39b4594eefbeca5404e1033758', 'request_id': 'req-e78a400f-07d8-4238-88fa-ddb8bf735277'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.308714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbaf9c39b4594eefbeca5404e1033758', 'request_id': 'req-e78a400f-07d8-4238-88fa-ddb8bf735277'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.319555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbaf9c39b4594eefbeca5404e1033758', 'request_id': 'req-e78a400f-07d8-4238-88fa-ddb8bf735277'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.336727 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] The token cache is being invalidate because project fbaf9c39b4594eefbeca5404e1033758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:15.337036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.337860 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.339985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1167/2318] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:15 2022] DELETE /identity/v3/projects/fbaf9c39b4594eefbeca5404e1033758 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:15.348599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:15.350707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:15.388139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932a48c-bece-4805-bf3c-eb2b206d16c1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:15.526587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:15.527380 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:15.527739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:15.528160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] PATH_INFO: `/v3/users/216d6257a13349cabc7fd5fd4e8af192` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:15.530130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216d6257a13349cabc7fd5fd4e8af192)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:15.532469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:15.626788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:15.639575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216d6257a13349cabc7fd5fd4e8af192'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.640615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216d6257a13349cabc7fd5fd4e8af192'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.654759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216d6257a13349cabc7fd5fd4e8af192'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.655988 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216d6257a13349cabc7fd5fd4e8af192'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.677125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427d325-f8a9-460a-993b-cbcdf35dabc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216d6257a13349cabc7fd5fd4e8af192'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:15.707659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1152/2319] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:15 2022] DELETE /identity/v3/users/216d6257a13349cabc7fd5fd4e8af192 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:15.880805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:15.883001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:15.909424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a400f-07d8-4238-88fa-ddb8bf735277 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:16.021911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:16.022445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:16.022799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:16.023110 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] PATH_INFO: `/v3/users/dd5fd430044c4fd39f08a5086bcb69ba` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:16.024923 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd5fd430044c4fd39f08a5086bcb69ba)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:16.027264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:16.135206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd5fd430044c4fd39f08a5086bcb69ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.155206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd5fd430044c4fd39f08a5086bcb69ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.157342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd5fd430044c4fd39f08a5086bcb69ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.159498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd5fd430044c4fd39f08a5086bcb69ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.181360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd5fd430044c4fd39f08a5086bcb69ba'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.211818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1168/2320] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:15 2022] DELETE /identity/v3/users/dd5fd430044c4fd39f08a5086bcb69ba => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:16.216582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76b910-1bd4-429a-8822-893f31d7b43b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:16.217072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76b910-1bd4-429a-8822-893f31d7b43b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:16.217757 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76b910-1bd4-429a-8822-893f31d7b43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:16.248939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ef76b910-1bd4-429a-8822-893f31d7b43b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:16.251027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ef76b910-1bd4-429a-8822-893f31d7b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:16.264945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:16.267081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:16.350378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:16.378257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c457a92-8e99-44b7-8ac9-c6a0b4541ae6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:16.385263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:16.386573 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:16.387547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:16.388050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] PATH_INFO: `/v3/projects/d883083f8a3c4f07a1208a4034c212fc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:16.389975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d883083f8a3c4f07a1208a4034c212fc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:16.392457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:16.407060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1153/2321] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:16.418852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76b910-1bd4-429a-8822-893f31d7b43b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:16.421083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ef76b910-1bd4-429a-8822-893f31d7b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:16.458178 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd883083f8a3c4f07a1208a4034c212fc', 'request_id': 'req-d487fc77-78b3-49df-9c49-9fb813771337'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.475570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd883083f8a3c4f07a1208a4034c212fc', 'request_id': 'req-d487fc77-78b3-49df-9c49-9fb813771337'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.490475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd883083f8a3c4f07a1208a4034c212fc', 'request_id': 'req-d487fc77-78b3-49df-9c49-9fb813771337'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.507129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] The token cache is being invalidate because project d883083f8a3c4f07a1208a4034c212fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:16.507423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.508684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d487fc77-78b3-49df-9c49-9fb813771337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:16.513534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76b910-1bd4-429a-8822-893f31d7b43b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:16.515842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1169/2322] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:16 2022] DELETE /identity/v3/projects/d883083f8a3c4f07a1208a4034c212fc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:16.521375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada3e2b-1f2d-426f-bbc0-83ece3d9816b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:16.521662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada3e2b-1f2d-426f-bbc0-83ece3d9816b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:16.521939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada3e2b-1f2d-426f-bbc0-83ece3d9816b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:16.546871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5ada3e2b-1f2d-426f-bbc0-83ece3d9816b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:16.548161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5ada3e2b-1f2d-426f-bbc0-83ece3d9816b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:16.582006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8906d7-927d-4974-936f-b11cdb77cd94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:16.582006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8906d7-927d-4974-936f-b11cdb77cd94 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:16.582291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8906d7-927d-4974-936f-b11cdb77cd94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:16.582897 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8906d7-927d-4974-936f-b11cdb77cd94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:16.584900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1a8906d7-927d-4974-936f-b11cdb77cd94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:16.625857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:16.627373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1170/2323] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:16.678401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8906d7-927d-4974-936f-b11cdb77cd94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:16.680008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8906d7-927d-4974-936f-b11cdb77cd94 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:16.691377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1154/2324] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:16.700474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f661ff8-fb2d-4743-a27e-ec82b6a886ec None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:16.700924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f661ff8-fb2d-4743-a27e-ec82b6a886ec None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:16.701431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f661ff8-fb2d-4743-a27e-ec82b6a886ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:16.724341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7f661ff8-fb2d-4743-a27e-ec82b6a886ec None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:16.726117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7f661ff8-fb2d-4743-a27e-ec82b6a886ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:16.739193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1171/2325] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:16.744396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8906d7-927d-4974-936f-b11cdb77cd94 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:16.759893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8906d7-927d-4974-936f-b11cdb77cd94 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:16.770719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9c0fa-5619-44c1-b010-3eea3999471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:16.772226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9c0fa-5619-44c1-b010-3eea3999471d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:16.772518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9c0fa-5619-44c1-b010-3eea3999471d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:16.774024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9c0fa-5619-44c1-b010-3eea3999471d admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:16.774239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9c0fa-5619-44c1-b010-3eea3999471d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:23:16.774685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9c0fa-5619-44c1-b010-3eea3999471d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:16.776299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd9c0fa-5619-44c1-b010-3eea3999471d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:16.793808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1155/2326] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:23:16.795684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f661ff8-fb2d-4743-a27e-ec82b6a886ec None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:16.806728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f661ff8-fb2d-4743-a27e-ec82b6a886ec None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:16.813179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066abe97-15aa-49fc-a0fc-8a55ba107699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:16.813806 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066abe97-15aa-49fc-a0fc-8a55ba107699 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:16.814085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066abe97-15aa-49fc-a0fc-8a55ba107699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:16.814359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066abe97-15aa-49fc-a0fc-8a55ba107699 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:16.814996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066abe97-15aa-49fc-a0fc-8a55ba107699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:16.817065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066abe97-15aa-49fc-a0fc-8a55ba107699 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:16.853893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1172/2327] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:23:16 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:16.856329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9c0fa-5619-44c1-b010-3eea3999471d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:16.877831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9c0fa-5619-44c1-b010-3eea3999471d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:16.948133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3567948-587a-462d-aab9-eb0bf88e769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:16.949025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3567948-587a-462d-aab9-eb0bf88e769e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:16.949453 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3567948-587a-462d-aab9-eb0bf88e769e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:16.949905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3567948-587a-462d-aab9-eb0bf88e769e admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:16.951252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3567948-587a-462d-aab9-eb0bf88e769e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:16.953412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3567948-587a-462d-aab9-eb0bf88e769e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:16.981108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1156/2328] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:23:16 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:16.985034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066abe97-15aa-49fc-a0fc-8a55ba107699 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:17.000293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066abe97-15aa-49fc-a0fc-8a55ba107699 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:17.007873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f86be-fa8e-4c73-bb43-e4135cf13d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:17.008579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f86be-fa8e-4c73-bb43-e4135cf13d1b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.008865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f86be-fa8e-4c73-bb43-e4135cf13d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.009267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f86be-fa8e-4c73-bb43-e4135cf13d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.010152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f86be-fa8e-4c73-bb43-e4135cf13d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:17.013884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f86be-fa8e-4c73-bb43-e4135cf13d1b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:17.028495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1173/2329] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:17.039661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3567948-587a-462d-aab9-eb0bf88e769e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:17.051440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3567948-587a-462d-aab9-eb0bf88e769e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:17.056945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ca0ae-aad8-4087-827a-f565199e88a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:17.057645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca0ae-aad8-4087-827a-f565199e88a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.058525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca0ae-aad8-4087-827a-f565199e88a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.058525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca0ae-aad8-4087-827a-f565199e88a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.058989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ca0ae-aad8-4087-827a-f565199e88a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:17.060775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ca0ae-aad8-4087-827a-f565199e88a8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:17.074821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1157/2330] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:17.077473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f86be-fa8e-4c73-bb43-e4135cf13d1b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:17.089283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f86be-fa8e-4c73-bb43-e4135cf13d1b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:17.096550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7f710-873e-41ac-9da1-198742d76875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:17.097360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7f710-873e-41ac-9da1-198742d76875 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.097839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7f710-873e-41ac-9da1-198742d76875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.098212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa7f710-873e-41ac-9da1-198742d76875 admin admin] PATH_INFO: `/v3/projects/1a86f263882a44f385804b84f83d735c/users/0c80648c3936461295821ce1431a4164/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.117926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7f710-873e-41ac-9da1-198742d76875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a86f263882a44f385804b84f83d735c, user_id=0c80648c3936461295821ce1431a4164, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:17.121726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa7f710-873e-41ac-9da1-198742d76875 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:17.147769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1174/2331] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:23:17 2022] PUT /identity/v3/projects/1a86f263882a44f385804b84f83d735c/users/0c80648c3936461295821ce1431a4164/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:17.158478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a86cf-75d5-4644-8945-5d7d3a3a6290 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.158773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a86cf-75d5-4644-8945-5d7d3a3a6290 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.159047 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a86cf-75d5-4644-8945-5d7d3a3a6290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.192989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cb8a86cf-75d5-4644-8945-5d7d3a3a6290 None None] MFA Rules not processed for user `0c80648c3936461295821ce1431a4164`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:17.194407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cb8a86cf-75d5-4644-8945-5d7d3a3a6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:17.239216 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7f710-873e-41ac-9da1-198742d76875 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:17.241728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9fa7f710-873e-41ac-9da1-198742d76875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:17.287838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:17.308550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1158/2332] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:17.329370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a86cf-75d5-4644-8945-5d7d3a3a6290 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:17.331106 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cb8a86cf-75d5-4644-8945-5d7d3a3a6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:17.368424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:17.397293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa7f710-873e-41ac-9da1-198742d76875 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:17.404738 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:17.405468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.405757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.406040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] PATH_INFO: `/v3/projects/c61099572cb04210988b5863247bc641` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.408051 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61099572cb04210988b5863247bc641)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:17.410066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:17.441760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a86cf-75d5-4644-8945-5d7d3a3a6290 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:17.447966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70463c-ed24-4734-b748-84ce1184b4b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:17.450390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70463c-ed24-4734-b748-84ce1184b4b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.451506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70463c-ed24-4734-b748-84ce1184b4b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.452026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70463c-ed24-4734-b748-84ce1184b4b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.469139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70463c-ed24-4734-b748-84ce1184b4b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:17.471033 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b70463c-ed24-4734-b748-84ce1184b4b8 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:17.499259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1159/2333] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:23:17.504605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61099572cb04210988b5863247bc641', 'request_id': 'req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:17.517781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61099572cb04210988b5863247bc641', 'request_id': 'req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:17.530825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61099572cb04210988b5863247bc641', 'request_id': 'req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:17.549265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] The token cache is being invalidate because project c61099572cb04210988b5863247bc641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:17.549919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:17.550938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f9d99ca3-d9ae-4a52-81c9-1020c8e00424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:17.556098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1175/2334] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:17 2022] DELETE /identity/v3/projects/c61099572cb04210988b5863247bc641 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:23:17.557755 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4eae17-7e30-4002-877c-f941d2243026 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.558048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4eae17-7e30-4002-877c-f941d2243026 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.558324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4eae17-7e30-4002-877c-f941d2243026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.584078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ce4eae17-7e30-4002-877c-f941d2243026 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:17.585442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ce4eae17-7e30-4002-877c-f941d2243026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:17.668904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1160/2335] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:17.672419 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d2d2c-812d-48aa-8491-ffc53f92a122 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.672664 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d2d2c-812d-48aa-8491-ffc53f92a122 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.673614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d2d2c-812d-48aa-8491-ffc53f92a122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.704652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-964d2d2c-812d-48aa-8491-ffc53f92a122 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:17.707396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-964d2d2c-812d-48aa-8491-ffc53f92a122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:17.725939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1176/2336] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:17.729486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4eae17-7e30-4002-877c-f941d2243026 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:17.751224 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4eae17-7e30-4002-877c-f941d2243026 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:17.765255 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9311fd-cf6c-4247-aa70-3d7f2fb9dfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:17.766404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9311fd-cf6c-4247-aa70-3d7f2fb9dfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.766404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9311fd-cf6c-4247-aa70-3d7f2fb9dfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.766576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9311fd-cf6c-4247-aa70-3d7f2fb9dfc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.767160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9311fd-cf6c-4247-aa70-3d7f2fb9dfc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:23:17.767464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9311fd-cf6c-4247-aa70-3d7f2fb9dfc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:17.769309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9311fd-cf6c-4247-aa70-3d7f2fb9dfc1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:17.789412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1161/2337] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:23:17.794035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d2d2c-812d-48aa-8491-ffc53f92a122 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:17.818555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d2d2c-812d-48aa-8491-ffc53f92a122 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:17.826528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738575b-fffd-4f5f-bd30-8db74ac8967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:17.827324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738575b-fffd-4f5f-bd30-8db74ac8967e admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.827487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738575b-fffd-4f5f-bd30-8db74ac8967e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.827808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738575b-fffd-4f5f-bd30-8db74ac8967e admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.828697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738575b-fffd-4f5f-bd30-8db74ac8967e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:17.831249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738575b-fffd-4f5f-bd30-8db74ac8967e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:17.871041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1177/2338] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:23:17 2022] POST /identity/v3/projects => generated 399 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:17.876987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9311fd-cf6c-4247-aa70-3d7f2fb9dfc1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:17.890175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9311fd-cf6c-4247-aa70-3d7f2fb9dfc1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:17.958784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114404d4-84f5-475f-8c63-e0d27eff01bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:17.959412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114404d4-84f5-475f-8c63-e0d27eff01bc admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:17.959679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114404d4-84f5-475f-8c63-e0d27eff01bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:17.959994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114404d4-84f5-475f-8c63-e0d27eff01bc admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:17.960635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114404d4-84f5-475f-8c63-e0d27eff01bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:17.962652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114404d4-84f5-475f-8c63-e0d27eff01bc admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:17.988453 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1162/2339] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:23:17 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:17.992315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738575b-fffd-4f5f-bd30-8db74ac8967e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:18.003781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738575b-fffd-4f5f-bd30-8db74ac8967e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:18.008873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1437c-60de-4329-a3c3-068334aaaf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:18.009166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1437c-60de-4329-a3c3-068334aaaf11 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:18.009448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1437c-60de-4329-a3c3-068334aaaf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:18.009722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1437c-60de-4329-a3c3-068334aaaf11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:18.010365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1437c-60de-4329-a3c3-068334aaaf11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:18.012519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1437c-60de-4329-a3c3-068334aaaf11 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:18.023592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1178/2340] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:23:18.027716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114404d4-84f5-475f-8c63-e0d27eff01bc admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:18.038731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114404d4-84f5-475f-8c63-e0d27eff01bc admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:18.043592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5328b0-53f1-4d5b-9f61-a7a1e8300ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:18.044362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5328b0-53f1-4d5b-9f61-a7a1e8300ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:18.045440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5328b0-53f1-4d5b-9f61-a7a1e8300ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:18.045839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5328b0-53f1-4d5b-9f61-a7a1e8300ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:18.046586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5328b0-53f1-4d5b-9f61-a7a1e8300ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:18.048414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5328b0-53f1-4d5b-9f61-a7a1e8300ee2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:18.062544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1163/2341] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:18.067291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1437c-60de-4329-a3c3-068334aaaf11 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:18.081408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1437c-60de-4329-a3c3-068334aaaf11 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:18.087474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b3770-bc22-4ed0-81b5-d5174a62efea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:18.088196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3770-bc22-4ed0-81b5-d5174a62efea admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:18.088477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3770-bc22-4ed0-81b5-d5174a62efea admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:18.088735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b3770-bc22-4ed0-81b5-d5174a62efea admin admin] PATH_INFO: `/v3/projects/49b3c8b3654a40d698b4cd5c3ea8ed21/users/39cd256873784abfa09628e778e8b44d/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:18.104909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b3770-bc22-4ed0-81b5-d5174a62efea admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b3c8b3654a40d698b4cd5c3ea8ed21, user_id=39cd256873784abfa09628e778e8b44d, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:18.108236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b3770-bc22-4ed0-81b5-d5174a62efea admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:18.121858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1179/2342] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:23:18 2022] PUT /identity/v3/projects/49b3c8b3654a40d698b4cd5c3ea8ed21/users/39cd256873784abfa09628e778e8b44d/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:18.133276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8dd356-1bd6-43ab-b590-a5d39d735528 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:18.133565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8dd356-1bd6-43ab-b590-a5d39d735528 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:18.133845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8dd356-1bd6-43ab-b590-a5d39d735528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:18.163673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fc8dd356-1bd6-43ab-b590-a5d39d735528 None None] MFA Rules not processed for user `39cd256873784abfa09628e778e8b44d`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:18.164935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fc8dd356-1bd6-43ab-b590-a5d39d735528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:18.182218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:18.250087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1164/2343] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:18.267163 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b3770-bc22-4ed0-81b5-d5174a62efea admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:18.269035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2a5b3770-bc22-4ed0-81b5-d5174a62efea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:18.331810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:18.351993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b3770-bc22-4ed0-81b5-d5174a62efea admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:18.355705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dad3f2-3c08-4f01-a526-d575ee689335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:18.356347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dad3f2-3c08-4f01-a526-d575ee689335 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:18.356628 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dad3f2-3c08-4f01-a526-d575ee689335 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:18.357454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dad3f2-3c08-4f01-a526-d575ee689335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:18.366097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dad3f2-3c08-4f01-a526-d575ee689335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:18.367903 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dad3f2-3c08-4f01-a526-d575ee689335 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:18.377802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1180/2344] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:20.053884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16825f-db22-4f9c-a66b-a44a8eabb57a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:20.054381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16825f-db22-4f9c-a66b-a44a8eabb57a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:20.054501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16825f-db22-4f9c-a66b-a44a8eabb57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:20.073730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-6e16825f-db22-4f9c-a66b-a44a8eabb57a None None] MFA Rules not processed for user `0c80648c3936461295821ce1431a4164`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:20.076081 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-6e16825f-db22-4f9c-a66b-a44a8eabb57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:20.134277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1165/2345] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:28.373123 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=9 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:28.673937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dad3f2-3c08-4f01-a526-d575ee689335 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:28.735498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dad3f2-3c08-4f01-a526-d575ee689335 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:28.740143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:28.740764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:28.746461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:28.746461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] PATH_INFO: `/v3/users/39cd256873784abfa09628e778e8b44d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:28.746461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39cd256873784abfa09628e778e8b44d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:28.746461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:28.804854 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39cd256873784abfa09628e778e8b44d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:28.816914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39cd256873784abfa09628e778e8b44d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:28.817601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39cd256873784abfa09628e778e8b44d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:28.818291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39cd256873784abfa09628e778e8b44d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:28.826530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39cd256873784abfa09628e778e8b44d'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:28.847580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1181/2346] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:28 2022] DELETE /identity/v3/users/39cd256873784abfa09628e778e8b44d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:28.851959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7f19-dcd4-42f1-a2c8-82213b6296dc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:28.852379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7f19-dcd4-42f1-a2c8-82213b6296dc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:28.852619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7f19-dcd4-42f1-a2c8-82213b6296dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:28.871851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f33c7f19-dcd4-42f1-a2c8-82213b6296dc None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:28.873123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f33c7f19-dcd4-42f1-a2c8-82213b6296dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:28.937425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1166/2347] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:23:28.947223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:28.948752 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:29.006262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e11a3c-01b6-4b23-8468-7ff364a74993 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:29.010048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c409b0c-c933-4ff8-a6ad-9a64e7c1309c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:29.010656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c409b0c-c933-4ff8-a6ad-9a64e7c1309c service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:29.010938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c409b0c-c933-4ff8-a6ad-9a64e7c1309c service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:29.011214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c409b0c-c933-4ff8-a6ad-9a64e7c1309c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:29.021329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c409b0c-c933-4ff8-a6ad-9a64e7c1309c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:29.022756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c409b0c-c933-4ff8-a6ad-9a64e7c1309c service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:29.032853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1182/2348] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:29.348672 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c7f19-dcd4-42f1-a2c8-82213b6296dc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:29.350123 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f33c7f19-dcd4-42f1-a2c8-82213b6296dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:29.363179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c7f19-dcd4-42f1-a2c8-82213b6296dc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:29.368497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:29.369141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:29.369427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:29.369703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] PATH_INFO: `/v3/projects/49b3c8b3654a40d698b4cd5c3ea8ed21` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:29.371663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b3c8b3654a40d698b4cd5c3ea8ed21)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:29.373610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:29.421368 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b3c8b3654a40d698b4cd5c3ea8ed21', 'request_id': 'req-b3610115-a0f1-436a-9786-0637ad2a3d59'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:29.427329 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b3c8b3654a40d698b4cd5c3ea8ed21', 'request_id': 'req-b3610115-a0f1-436a-9786-0637ad2a3d59'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:29.429850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:29.435117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b3c8b3654a40d698b4cd5c3ea8ed21', 'request_id': 'req-b3610115-a0f1-436a-9786-0637ad2a3d59'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:29.446867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] The token cache is being invalidate because project 49b3c8b3654a40d698b4cd5c3ea8ed21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:29.447162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:29.447810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:29.449288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1167/2349] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:29 2022] DELETE /identity/v3/projects/49b3c8b3654a40d698b4cd5c3ea8ed21 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:23:39.028036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:39.443839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=3,keystone:SELECT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:53.387716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc4592-4dac-42af-b65a-db21b7fe02dc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:53.388242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc4592-4dac-42af-b65a-db21b7fe02dc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:53.388492 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc4592-4dac-42af-b65a-db21b7fe02dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:53.411164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3fdc4592-4dac-42af-b65a-db21b7fe02dc None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:53.412490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3fdc4592-4dac-42af-b65a-db21b7fe02dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:53.468769 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1183/2350] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:23:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:23:53.516370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:53.517830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:53.589097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3610115-a0f1-436a-9786-0637ad2a3d59 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:53.594963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe528c-0581-4639-bce9-776936ab5846 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:53.595621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe528c-0581-4639-bce9-776936ab5846 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:53.595942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe528c-0581-4639-bce9-776936ab5846 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:53.596217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe528c-0581-4639-bce9-776936ab5846 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:53.604477 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe528c-0581-4639-bce9-776936ab5846 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:53.606041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe528c-0581-4639-bce9-776936ab5846 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:53.614369 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1168/2351] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:23:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:23:56.098472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc4592-4dac-42af-b65a-db21b7fe02dc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:56.100607 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3fdc4592-4dac-42af-b65a-db21b7fe02dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:56.201847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc4592-4dac-42af-b65a-db21b7fe02dc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:56.212264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328de34-2c20-4a85-8b06-92dffea91610 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:56.213174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328de34-2c20-4a85-8b06-92dffea91610 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:56.213554 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328de34-2c20-4a85-8b06-92dffea91610 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:56.213925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328de34-2c20-4a85-8b06-92dffea91610 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:56.218172 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c328de34-2c20-4a85-8b06-92dffea91610 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:56.236634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328de34-2c20-4a85-8b06-92dffea91610 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:56.238333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328de34-2c20-4a85-8b06-92dffea91610 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:56.261345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1184/2352] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:57.048442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe528c-0581-4639-bce9-776936ab5846 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:57.050137 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-57fe528c-0581-4639-bce9-776936ab5846 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:57.151328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe528c-0581-4639-bce9-776936ab5846 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:57.157277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:57.158012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:57.158367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:57.158777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] PATH_INFO: `/v3/users/85748c19df46423b9eff962e22396c67` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:57.159983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85748c19df46423b9eff962e22396c67)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:57.162571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:57.238131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85748c19df46423b9eff962e22396c67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:57.238874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85748c19df46423b9eff962e22396c67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:57.252446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85748c19df46423b9eff962e22396c67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:57.253340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85748c19df46423b9eff962e22396c67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:57.264784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85748c19df46423b9eff962e22396c67'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:57.285126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1169/2353] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:23:57 2022] DELETE /identity/v3/users/85748c19df46423b9eff962e22396c67 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:57.289798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcf0fa-0aee-40a1-8d95-a78af91bfc02 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:57.290635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcf0fa-0aee-40a1-8d95-a78af91bfc02 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:57.290912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcf0fa-0aee-40a1-8d95-a78af91bfc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:57.314797 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fbdcf0fa-0aee-40a1-8d95-a78af91bfc02 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:57.316601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fbdcf0fa-0aee-40a1-8d95-a78af91bfc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:57.407798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1185/2354] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:57.415736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:57.417663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:57.430363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:57.499669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6589b08-714e-45bf-bd29-5cba01f03b09 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:57.504226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c2d30-fe99-4aa7-bb5d-541e3113d775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:57.504603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c2d30-fe99-4aa7-bb5d-541e3113d775 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:57.504958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c2d30-fe99-4aa7-bb5d-541e3113d775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:57.505345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c2d30-fe99-4aa7-bb5d-541e3113d775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:57.516617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c2d30-fe99-4aa7-bb5d-541e3113d775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:57.518153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c2d30-fe99-4aa7-bb5d-541e3113d775 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:57.538819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1170/2355] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:23:57.893604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcf0fa-0aee-40a1-8d95-a78af91bfc02 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:57.895472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fbdcf0fa-0aee-40a1-8d95-a78af91bfc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:57.917574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcf0fa-0aee-40a1-8d95-a78af91bfc02 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:57.931588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:57.932394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:57.932735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:57.933013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] PATH_INFO: `/v3/projects/19e7745bc1df472aa25467a3e4f47602` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:57.934266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19e7745bc1df472aa25467a3e4f47602)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:57.937137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:57.981332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:57.998141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19e7745bc1df472aa25467a3e4f47602', 'request_id': 'req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:58.006113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19e7745bc1df472aa25467a3e4f47602', 'request_id': 'req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:58.029551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19e7745bc1df472aa25467a3e4f47602', 'request_id': 'req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:58.053842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] The token cache is being invalidate because project 19e7745bc1df472aa25467a3e4f47602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:23:58.054398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:58.055642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e0ba8a1-ab4d-488d-ad50-a04adb53f251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:23:58.058016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1186/2356] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:23:57 2022] DELETE /identity/v3/projects/19e7745bc1df472aa25467a3e4f47602 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:58.062585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba25c8-53df-4a9e-83c5-df75bf3147dc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.063077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba25c8-53df-4a9e-83c5-df75bf3147dc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.063188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba25c8-53df-4a9e-83c5-df75bf3147dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:58.087452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d8ba25c8-53df-4a9e-83c5-df75bf3147dc None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:58.090362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d8ba25c8-53df-4a9e-83c5-df75bf3147dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:58.202295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1171/2357] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:58.212386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f929af-0e69-416c-a73b-d79d7abadcd4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.212911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f929af-0e69-416c-a73b-d79d7abadcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.213496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f929af-0e69-416c-a73b-d79d7abadcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:58.279260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-81f929af-0e69-416c-a73b-d79d7abadcd4 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:58.280960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-81f929af-0e69-416c-a73b-d79d7abadcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:58.304933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1187/2358] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:58.308805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba25c8-53df-4a9e-83c5-df75bf3147dc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:58.323191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba25c8-53df-4a9e-83c5-df75bf3147dc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:58.341160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bbffa-cd92-4f6e-b2b6-9aecdcd31e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:58.343930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bbffa-cd92-4f6e-b2b6-9aecdcd31e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.345443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bbffa-cd92-4f6e-b2b6-9aecdcd31e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.345888 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bbffa-cd92-4f6e-b2b6-9aecdcd31e54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:58.347697 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bbffa-cd92-4f6e-b2b6-9aecdcd31e54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:23:58.350502 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bbffa-cd92-4f6e-b2b6-9aecdcd31e54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:58.352486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bbffa-cd92-4f6e-b2b6-9aecdcd31e54 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:58.373197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1172/2359] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:23:58.374920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f929af-0e69-416c-a73b-d79d7abadcd4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:58.396963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f929af-0e69-416c-a73b-d79d7abadcd4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:58.404325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c0096-36dd-4867-8393-8a05323ff2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:58.405470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c0096-36dd-4867-8393-8a05323ff2bc admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.405965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c0096-36dd-4867-8393-8a05323ff2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.406445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c0096-36dd-4867-8393-8a05323ff2bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:58.407341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c0096-36dd-4867-8393-8a05323ff2bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:58.409931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c0096-36dd-4867-8393-8a05323ff2bc admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:58.467719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1188/2360] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:23:58 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:23:58.470147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bbffa-cd92-4f6e-b2b6-9aecdcd31e54 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:58.484044 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bbffa-cd92-4f6e-b2b6-9aecdcd31e54 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:58.560967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d3fe4-0bd3-4dce-a8df-c96bfced6397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:58.561788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d3fe4-0bd3-4dce-a8df-c96bfced6397 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.562194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d3fe4-0bd3-4dce-a8df-c96bfced6397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.562471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d3fe4-0bd3-4dce-a8df-c96bfced6397 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:58.563124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d3fe4-0bd3-4dce-a8df-c96bfced6397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:58.565325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d3fe4-0bd3-4dce-a8df-c96bfced6397 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:58.600252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1173/2361] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:23:58 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:23:58.604364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c0096-36dd-4867-8393-8a05323ff2bc admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:58.621420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c0096-36dd-4867-8393-8a05323ff2bc admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:58.631593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764d29d-61c4-429e-bf4e-b38cc0a25e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:58.632412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764d29d-61c4-429e-bf4e-b38cc0a25e3a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.633049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764d29d-61c4-429e-bf4e-b38cc0a25e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.633049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764d29d-61c4-429e-bf4e-b38cc0a25e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:58.634130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764d29d-61c4-429e-bf4e-b38cc0a25e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:58.636822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764d29d-61c4-429e-bf4e-b38cc0a25e3a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:58.648531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1189/2362] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:58.652679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d3fe4-0bd3-4dce-a8df-c96bfced6397 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:58.675078 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d3fe4-0bd3-4dce-a8df-c96bfced6397 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:58.680565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48053d-0979-47f2-8a53-3404b316bf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:58.681387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48053d-0979-47f2-8a53-3404b316bf8b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.681857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48053d-0979-47f2-8a53-3404b316bf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.682294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48053d-0979-47f2-8a53-3404b316bf8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:58.683104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48053d-0979-47f2-8a53-3404b316bf8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:58.684915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48053d-0979-47f2-8a53-3404b316bf8b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:58.706059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1174/2363] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:23:58.710823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764d29d-61c4-429e-bf4e-b38cc0a25e3a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:58.725050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764d29d-61c4-429e-bf4e-b38cc0a25e3a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:58.732608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df258d98-a6c4-426f-a3e2-61a39fdea331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:58.733659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df258d98-a6c4-426f-a3e2-61a39fdea331 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.734108 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df258d98-a6c4-426f-a3e2-61a39fdea331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.734579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df258d98-a6c4-426f-a3e2-61a39fdea331 admin admin] PATH_INFO: `/v3/projects/66219614f7924f03ac83f5e8f659763b/users/f4eff2774ce94fc8bd1d872faf0efabb/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:58.756357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df258d98-a6c4-426f-a3e2-61a39fdea331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66219614f7924f03ac83f5e8f659763b, user_id=f4eff2774ce94fc8bd1d872faf0efabb, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:58.760514 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df258d98-a6c4-426f-a3e2-61a39fdea331 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:58.780132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1190/2364] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:23:58 2022] PUT /identity/v3/projects/66219614f7924f03ac83f5e8f659763b/users/f4eff2774ce94fc8bd1d872faf0efabb/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:23:58.790273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09701a8-8d93-40f9-86a7-dae00628e07f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.790466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09701a8-8d93-40f9-86a7-dae00628e07f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.790742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09701a8-8d93-40f9-86a7-dae00628e07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:58.820461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a09701a8-8d93-40f9-86a7-dae00628e07f None None] MFA Rules not processed for user `f4eff2774ce94fc8bd1d872faf0efabb`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:23:58.821686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a09701a8-8d93-40f9-86a7-dae00628e07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:58.828418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:23:58.906551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1175/2365] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:23:58.918573 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df258d98-a6c4-426f-a3e2-61a39fdea331 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:23:58.920370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-df258d98-a6c4-426f-a3e2-61a39fdea331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:23:58.992031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df258d98-a6c4-426f-a3e2-61a39fdea331 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:23:58.996798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66856405-e512-41f8-9d22-be2467aea746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:23:58.997437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66856405-e512-41f8-9d22-be2467aea746 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:23:58.997759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66856405-e512-41f8-9d22-be2467aea746 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:23:58.998034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66856405-e512-41f8-9d22-be2467aea746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:23:59.008067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66856405-e512-41f8-9d22-be2467aea746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:23:59.009686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66856405-e512-41f8-9d22-be2467aea746 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:23:59.020796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1191/2366] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:23:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:24:01.748886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09701a8-8d93-40f9-86a7-dae00628e07f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:01.833784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09701a8-8d93-40f9-86a7-dae00628e07f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:01.839024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:01.839747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:01.840146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:01.840494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] PATH_INFO: `/v3/users/f4eff2774ce94fc8bd1d872faf0efabb` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:01.842257 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4eff2774ce94fc8bd1d872faf0efabb)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:01.844548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:01.914001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4eff2774ce94fc8bd1d872faf0efabb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:01.915121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4eff2774ce94fc8bd1d872faf0efabb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:01.928857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4eff2774ce94fc8bd1d872faf0efabb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:01.930611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4eff2774ce94fc8bd1d872faf0efabb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:01.943985 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4eff2774ce94fc8bd1d872faf0efabb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:01.989383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1176/2367] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:24:01 2022] DELETE /identity/v3/users/f4eff2774ce94fc8bd1d872faf0efabb => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:01.990677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f6a9b-b490-4841-93ac-388d5eedb26f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:01.991200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f6a9b-b490-4841-93ac-388d5eedb26f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:01.991200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f6a9b-b490-4841-93ac-388d5eedb26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:02.018351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ff4f6a9b-b490-4841-93ac-388d5eedb26f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:02.019596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ff4f6a9b-b490-4841-93ac-388d5eedb26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:02.118768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:02.145734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1192/2368] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:24:02.183093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:02.185435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:02.234742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:02.291267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d9bdb-59dc-4331-a250-03e94414bdf9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:02.296173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d61782-fe05-46e1-9686-4bb7c94aa124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:02.296789 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d61782-fe05-46e1-9686-4bb7c94aa124 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:02.297084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d61782-fe05-46e1-9686-4bb7c94aa124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:02.297343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d61782-fe05-46e1-9686-4bb7c94aa124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:02.307148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d61782-fe05-46e1-9686-4bb7c94aa124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:02.308616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d61782-fe05-46e1-9686-4bb7c94aa124 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:02.344028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1177/2369] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:24:02.805660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f6a9b-b490-4841-93ac-388d5eedb26f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:02.809034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ff4f6a9b-b490-4841-93ac-388d5eedb26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:02.826937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f6a9b-b490-4841-93ac-388d5eedb26f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:02.832456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:02.833119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:02.833402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:02.833734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] PATH_INFO: `/v3/projects/66219614f7924f03ac83f5e8f659763b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:02.834944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=66219614f7924f03ac83f5e8f659763b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:02.837087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:02.897782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66219614f7924f03ac83f5e8f659763b', 'request_id': 'req-9875c599-e321-451d-bfb6-41890af9a8ef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:02.904871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66219614f7924f03ac83f5e8f659763b', 'request_id': 'req-9875c599-e321-451d-bfb6-41890af9a8ef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:02.920968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66219614f7924f03ac83f5e8f659763b', 'request_id': 'req-9875c599-e321-451d-bfb6-41890af9a8ef'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:02.937254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] The token cache is being invalidate because project 66219614f7924f03ac83f5e8f659763b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:24:02.937551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:02.938402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9875c599-e321-451d-bfb6-41890af9a8ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:02.940483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1193/2370] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:24:02 2022] DELETE /identity/v3/projects/66219614f7924f03ac83f5e8f659763b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:02.945401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dffb61-9b18-4b99-950d-d272a1eeef8c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:02.945898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dffb61-9b18-4b99-950d-d272a1eeef8c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:02.945995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dffb61-9b18-4b99-950d-d272a1eeef8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:02.968832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-66dffb61-9b18-4b99-950d-d272a1eeef8c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:02.970059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-66dffb61-9b18-4b99-950d-d272a1eeef8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:03.038112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1178/2371] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:24:03.045812 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe961ab-2c6e-4b78-a803-bd033310ffd3 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.046789 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe961ab-2c6e-4b78-a803-bd033310ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.047215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe961ab-2c6e-4b78-a803-bd033310ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.070132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bfe961ab-2c6e-4b78-a803-bd033310ffd3 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:03.071496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bfe961ab-2c6e-4b78-a803-bd033310ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:03.080832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1194/2372] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:24:03.085126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dffb61-9b18-4b99-950d-d272a1eeef8c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:03.095845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dffb61-9b18-4b99-950d-d272a1eeef8c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:03.101402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02020661-51bd-46f3-a7ab-bda073617e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:03.102220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02020661-51bd-46f3-a7ab-bda073617e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.102546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02020661-51bd-46f3-a7ab-bda073617e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.102829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02020661-51bd-46f3-a7ab-bda073617e6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.103578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02020661-51bd-46f3-a7ab-bda073617e6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:24:03.103954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02020661-51bd-46f3-a7ab-bda073617e6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:03.105719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02020661-51bd-46f3-a7ab-bda073617e6f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:03.124742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1179/2373] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:24:03.129605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe961ab-2c6e-4b78-a803-bd033310ffd3 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:03.140715 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe961ab-2c6e-4b78-a803-bd033310ffd3 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:03.147127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea31be9-e967-4ef6-a6e1-488dd4474ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:03.147774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea31be9-e967-4ef6-a6e1-488dd4474ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.148057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea31be9-e967-4ef6-a6e1-488dd4474ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.148342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea31be9-e967-4ef6-a6e1-488dd4474ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.148973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea31be9-e967-4ef6-a6e1-488dd4474ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:03.150976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea31be9-e967-4ef6-a6e1-488dd4474ab1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:03.182511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1195/2374] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:24:03 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:24:03.187915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02020661-51bd-46f3-a7ab-bda073617e6f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:03.203013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02020661-51bd-46f3-a7ab-bda073617e6f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:03.286388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561f76c-f755-4372-924b-2ff3c0507306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:03.287383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561f76c-f755-4372-924b-2ff3c0507306 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.287928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561f76c-f755-4372-924b-2ff3c0507306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.288275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561f76c-f755-4372-924b-2ff3c0507306 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.289596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561f76c-f755-4372-924b-2ff3c0507306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:03.291834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561f76c-f755-4372-924b-2ff3c0507306 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:03.317380 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1180/2375] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:24:03 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:24:03.321989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea31be9-e967-4ef6-a6e1-488dd4474ab1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:03.335880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea31be9-e967-4ef6-a6e1-488dd4474ab1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:03.341059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d9196-b603-4a55-b30e-7b06961133ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:03.344428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d9196-b603-4a55-b30e-7b06961133ed admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.344428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d9196-b603-4a55-b30e-7b06961133ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.344428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d9196-b603-4a55-b30e-7b06961133ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.344428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d9196-b603-4a55-b30e-7b06961133ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:03.345346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d9196-b603-4a55-b30e-7b06961133ed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:03.357350 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1196/2376] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:24:03.361420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561f76c-f755-4372-924b-2ff3c0507306 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:03.389376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561f76c-f755-4372-924b-2ff3c0507306 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:03.413399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd135369-a60f-4fa6-8364-b262a96bf602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:03.414142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135369-a60f-4fa6-8364-b262a96bf602 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.414497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135369-a60f-4fa6-8364-b262a96bf602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.415321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd135369-a60f-4fa6-8364-b262a96bf602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.416859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd135369-a60f-4fa6-8364-b262a96bf602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:03.418371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd135369-a60f-4fa6-8364-b262a96bf602 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:03.433777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1181/2377] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:24:03.438501 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d9196-b603-4a55-b30e-7b06961133ed admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:03.459039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d9196-b603-4a55-b30e-7b06961133ed admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:03.467988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ffac3-6c92-40e6-90f7-3f3f92c44554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:03.467988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ffac3-6c92-40e6-90f7-3f3f92c44554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.467988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ffac3-6c92-40e6-90f7-3f3f92c44554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.467988 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ffac3-6c92-40e6-90f7-3f3f92c44554 admin admin] PATH_INFO: `/v3/projects/89bf4044ba70487fa77f708effd682c6/users/2e7c334e7a5d4a2db18b35dc5ab2c48e/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.508131 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ffac3-6c92-40e6-90f7-3f3f92c44554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89bf4044ba70487fa77f708effd682c6, user_id=2e7c334e7a5d4a2db18b35dc5ab2c48e, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:03.510892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ffac3-6c92-40e6-90f7-3f3f92c44554 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:03.550353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1197/2378] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:24:03 2022] PUT /identity/v3/projects/89bf4044ba70487fa77f708effd682c6/users/2e7c334e7a5d4a2db18b35dc5ab2c48e/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:03.563132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e09f53-d879-4008-a4a1-3be5dde7f39c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.563659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e09f53-d879-4008-a4a1-3be5dde7f39c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.564102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e09f53-d879-4008-a4a1-3be5dde7f39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.601998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e5e09f53-d879-4008-a4a1-3be5dde7f39c None None] MFA Rules not processed for user `2e7c334e7a5d4a2db18b35dc5ab2c48e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:03.603298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e5e09f53-d879-4008-a4a1-3be5dde7f39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:03.641233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:03.704587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1182/2379] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:24:03.712320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ffac3-6c92-40e6-90f7-3f3f92c44554 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:03.714096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c62ffac3-6c92-40e6-90f7-3f3f92c44554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:03.815714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ffac3-6c92-40e6-90f7-3f3f92c44554 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:03.820200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a9b66-af55-48d4-aa5d-907d8d95d615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:03.820915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a9b66-af55-48d4-aa5d-907d8d95d615 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.821261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a9b66-af55-48d4-aa5d-907d8d95d615 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.821609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a9b66-af55-48d4-aa5d-907d8d95d615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.837003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a9b66-af55-48d4-aa5d-907d8d95d615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:03.838577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a9b66-af55-48d4-aa5d-907d8d95d615 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:03.850083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1198/2380] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:24:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:24:03.881694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e09f53-d879-4008-a4a1-3be5dde7f39c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:03.963376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e09f53-d879-4008-a4a1-3be5dde7f39c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:03.970506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:03.971402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:03.971887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:03.972469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] PATH_INFO: `/v3/users/2e7c334e7a5d4a2db18b35dc5ab2c48e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:03.974125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7c334e7a5d4a2db18b35dc5ab2c48e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:03.977394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:04.040974 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7c334e7a5d4a2db18b35dc5ab2c48e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.041644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7c334e7a5d4a2db18b35dc5ab2c48e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.056610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7c334e7a5d4a2db18b35dc5ab2c48e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.057771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7c334e7a5d4a2db18b35dc5ab2c48e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.066802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7c334e7a5d4a2db18b35dc5ab2c48e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.093718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1183/2381] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:24:03 2022] DELETE /identity/v3/users/2e7c334e7a5d4a2db18b35dc5ab2c48e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:24:04.099782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab2b00-31fb-485f-ae84-eddd5d84ac06 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:04.100471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab2b00-31fb-485f-ae84-eddd5d84ac06 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:04.100471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab2b00-31fb-485f-ae84-eddd5d84ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:04.122711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-89ab2b00-31fb-485f-ae84-eddd5d84ac06 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:04.124595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-89ab2b00-31fb-485f-ae84-eddd5d84ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:04.139353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:04.198859 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1199/2382] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:04.209928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:04.213343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:04.277941 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:04.289380 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a6c74-1d4c-4fa8-a4f3-be9064fd986d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:04.295382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d9ccf-fe5c-4b0f-8e78-76b4428cb330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:04.297099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d9ccf-fe5c-4b0f-8e78-76b4428cb330 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:04.297461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d9ccf-fe5c-4b0f-8e78-76b4428cb330 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:04.297853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d9ccf-fe5c-4b0f-8e78-76b4428cb330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:04.308924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d9ccf-fe5c-4b0f-8e78-76b4428cb330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:04.310497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d9ccf-fe5c-4b0f-8e78-76b4428cb330 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:04.321131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1184/2383] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:24:04.700400 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab2b00-31fb-485f-ae84-eddd5d84ac06 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:04.702065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-89ab2b00-31fb-485f-ae84-eddd5d84ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:04.719137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab2b00-31fb-485f-ae84-eddd5d84ac06 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:04.733250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:04.734429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:04.734820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:04.735244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] PATH_INFO: `/v3/projects/89bf4044ba70487fa77f708effd682c6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:04.736601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89bf4044ba70487fa77f708effd682c6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:04.739225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:04.894006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89bf4044ba70487fa77f708effd682c6', 'request_id': 'req-9ce86beb-ca21-4771-a824-f2af54caa3c4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.901881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89bf4044ba70487fa77f708effd682c6', 'request_id': 'req-9ce86beb-ca21-4771-a824-f2af54caa3c4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.913143 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89bf4044ba70487fa77f708effd682c6', 'request_id': 'req-9ce86beb-ca21-4771-a824-f2af54caa3c4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.934611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] The token cache is being invalidate because project 89bf4044ba70487fa77f708effd682c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:24:04.935483 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.936295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-9ce86beb-ca21-4771-a824-f2af54caa3c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:04.937884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1200/2384] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:24:04 2022] DELETE /identity/v3/projects/89bf4044ba70487fa77f708effd682c6 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:04.943428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd65a2-284d-4586-9167-2d4ed859add9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:04.944021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd65a2-284d-4586-9167-2d4ed859add9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:04.944508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd65a2-284d-4586-9167-2d4ed859add9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:04.980951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-eacd65a2-284d-4586-9167-2d4ed859add9 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:04.985020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eacd65a2-284d-4586-9167-2d4ed859add9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:05.083086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1185/2385] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:24:05.090758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bc81c-ce4c-46f3-b07d-51c24ff8cdfb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:05.091104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bc81c-ce4c-46f3-b07d-51c24ff8cdfb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:05.091347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bc81c-ce4c-46f3-b07d-51c24ff8cdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:05.134613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-609bc81c-ce4c-46f3-b07d-51c24ff8cdfb None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:05.136550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-609bc81c-ce4c-46f3-b07d-51c24ff8cdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:05.148595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1201/2386] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:05.154192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd65a2-284d-4586-9167-2d4ed859add9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:05.172653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd65a2-284d-4586-9167-2d4ed859add9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:05.183886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31d731-3de9-4f65-9005-04a067e89ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:05.185798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31d731-3de9-4f65-9005-04a067e89ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:05.186188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31d731-3de9-4f65-9005-04a067e89ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:05.186624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31d731-3de9-4f65-9005-04a067e89ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:05.187512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31d731-3de9-4f65-9005-04a067e89ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:24:05.190748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31d731-3de9-4f65-9005-04a067e89ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:05.192926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31d731-3de9-4f65-9005-04a067e89ff7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:05.222463 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1186/2387] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:24:05.224320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bc81c-ce4c-46f3-b07d-51c24ff8cdfb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:05.233795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bc81c-ce4c-46f3-b07d-51c24ff8cdfb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:05.242946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807564-ba30-4b8e-b575-378ff8ee4d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:05.243732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807564-ba30-4b8e-b575-378ff8ee4d71 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:05.244156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807564-ba30-4b8e-b575-378ff8ee4d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:05.244534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1807564-ba30-4b8e-b575-378ff8ee4d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:05.245276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1807564-ba30-4b8e-b575-378ff8ee4d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:05.247950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1807564-ba30-4b8e-b575-378ff8ee4d71 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:05.288623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1202/2388] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:24:05 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:24:05.290704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31d731-3de9-4f65-9005-04a067e89ff7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:05.303543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31d731-3de9-4f65-9005-04a067e89ff7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:05.438139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f089f-882e-49ef-a105-4aba0e544b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:05.450084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f089f-882e-49ef-a105-4aba0e544b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:05.451991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f089f-882e-49ef-a105-4aba0e544b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:05.453689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f089f-882e-49ef-a105-4aba0e544b4e admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:05.455231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f089f-882e-49ef-a105-4aba0e544b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:05.459163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f089f-882e-49ef-a105-4aba0e544b4e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:05.525079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1187/2389] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:24:05 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:24:05.529213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807564-ba30-4b8e-b575-378ff8ee4d71 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:05.541626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1807564-ba30-4b8e-b575-378ff8ee4d71 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:05.546570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a6302-6e69-4abd-b511-50af23c92ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:05.547197 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a6302-6e69-4abd-b511-50af23c92ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:05.547490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a6302-6e69-4abd-b511-50af23c92ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:05.547935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a6302-6e69-4abd-b511-50af23c92ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:05.548586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a6302-6e69-4abd-b511-50af23c92ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:05.550330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a6302-6e69-4abd-b511-50af23c92ea9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:05.569061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1203/2390] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:24:05.573568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f089f-882e-49ef-a105-4aba0e544b4e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:05.590267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f089f-882e-49ef-a105-4aba0e544b4e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:05.595836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069d2fb-92ae-42fb-9ecb-5de441560728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:05.596674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069d2fb-92ae-42fb-9ecb-5de441560728 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:05.597305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069d2fb-92ae-42fb-9ecb-5de441560728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:05.597305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069d2fb-92ae-42fb-9ecb-5de441560728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:05.598060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069d2fb-92ae-42fb-9ecb-5de441560728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:05.599782 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069d2fb-92ae-42fb-9ecb-5de441560728 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:05.614096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1188/2391] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:24:05.618208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a6302-6e69-4abd-b511-50af23c92ea9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:05.632766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a6302-6e69-4abd-b511-50af23c92ea9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:05.637841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ec6d1-aa9b-447c-8440-36f4ef2ea00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:05.638571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ec6d1-aa9b-447c-8440-36f4ef2ea00e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:05.638845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ec6d1-aa9b-447c-8440-36f4ef2ea00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:05.639157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ec6d1-aa9b-447c-8440-36f4ef2ea00e admin admin] PATH_INFO: `/v3/projects/9de710fda38c429a91595545e9a74dcb/users/8e0b6c8f5ec24d478f47a3d8d2fd941e/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:05.657640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ec6d1-aa9b-447c-8440-36f4ef2ea00e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9de710fda38c429a91595545e9a74dcb, user_id=8e0b6c8f5ec24d478f47a3d8d2fd941e, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:05.660602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ec6d1-aa9b-447c-8440-36f4ef2ea00e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:05.677717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1204/2392] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:24:05 2022] PUT /identity/v3/projects/9de710fda38c429a91595545e9a74dcb/users/8e0b6c8f5ec24d478f47a3d8d2fd941e/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:05.683025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb70fec-b89a-49f5-87de-d23f3d56211c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:05.683314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb70fec-b89a-49f5-87de-d23f3d56211c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:05.683585 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb70fec-b89a-49f5-87de-d23f3d56211c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:05.709151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5fb70fec-b89a-49f5-87de-d23f3d56211c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:05.710537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5fb70fec-b89a-49f5-87de-d23f3d56211c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:05.779457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:05.797120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1189/2393] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:05.808468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ec6d1-aa9b-447c-8440-36f4ef2ea00e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:05.810567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1e9ec6d1-aa9b-447c-8440-36f4ef2ea00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:05.891545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:05.916886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ec6d1-aa9b-447c-8440-36f4ef2ea00e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:05.923683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311e04-fd7d-49fa-9d73-9bd0aa14be01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:05.924478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311e04-fd7d-49fa-9d73-9bd0aa14be01 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:05.924763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311e04-fd7d-49fa-9d73-9bd0aa14be01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:05.925759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311e04-fd7d-49fa-9d73-9bd0aa14be01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:05.940733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1311e04-fd7d-49fa-9d73-9bd0aa14be01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:05.943117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1311e04-fd7d-49fa-9d73-9bd0aa14be01 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:05.957155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1205/2394] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:24:07.621880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce591f18-c04a-4e93-b465-dfae819811d5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:07.621880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce591f18-c04a-4e93-b465-dfae819811d5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:07.623158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce591f18-c04a-4e93-b465-dfae819811d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:07.660415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ce591f18-c04a-4e93-b465-dfae819811d5 None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:07.661770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ce591f18-c04a-4e93-b465-dfae819811d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:07.738577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1190/2395] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:24:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:07.755660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311e04-fd7d-49fa-9d73-9bd0aa14be01 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:07.761794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e1311e04-fd7d-49fa-9d73-9bd0aa14be01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:07.864244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311e04-fd7d-49fa-9d73-9bd0aa14be01 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:07.872106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8723fd-35f7-476d-8374-c12ce7b62b5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:07.873164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8723fd-35f7-476d-8374-c12ce7b62b5c service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:07.874058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8723fd-35f7-476d-8374-c12ce7b62b5c service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:07.874662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8723fd-35f7-476d-8374-c12ce7b62b5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:07.889035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8723fd-35f7-476d-8374-c12ce7b62b5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:07.892036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8723fd-35f7-476d-8374-c12ce7b62b5c service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:07.910562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1206/2396] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:24:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:24:08.967748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d4b75-6f51-47a2-94ed-6cb8ba105882 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:08.968309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d4b75-6f51-47a2-94ed-6cb8ba105882 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:08.968394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d4b75-6f51-47a2-94ed-6cb8ba105882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:09.017396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-ec7d4b75-6f51-47a2-94ed-6cb8ba105882 None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:09.018707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ec7d4b75-6f51-47a2-94ed-6cb8ba105882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:09.038257 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1191/2397] 10.222.0.71 () {58 vars in 967 bytes} [Wed Sep 21 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:24:09.054810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8723fd-35f7-476d-8374-c12ce7b62b5c service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:09.071663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8723fd-35f7-476d-8374-c12ce7b62b5c service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:09.084970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd2c46-da3a-4910-bd01-030e1200c822 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:09.085647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd2c46-da3a-4910-bd01-030e1200c822 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:09.086125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd2c46-da3a-4910-bd01-030e1200c822 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:09.090308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd2c46-da3a-4910-bd01-030e1200c822 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:09.111152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd2c46-da3a-4910-bd01-030e1200c822 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:09.113159 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd2c46-da3a-4910-bd01-030e1200c822 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:09.125475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1207/2398] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:24:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:24:09.657032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515ed90-4bda-4135-a4b0-5b8367fc2e32 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:09.658299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515ed90-4bda-4135-a4b0-5b8367fc2e32 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:09.658744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515ed90-4bda-4135-a4b0-5b8367fc2e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:09.700035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7515ed90-4bda-4135-a4b0-5b8367fc2e32 None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:09.701526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7515ed90-4bda-4135-a4b0-5b8367fc2e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:09.710925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1192/2399] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:09.725999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd2c46-da3a-4910-bd01-030e1200c822 service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:09.741351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd2c46-da3a-4910-bd01-030e1200c822 service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:09.759822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457b197-2068-4f97-97aa-8e9e8d087072 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:09.760460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457b197-2068-4f97-97aa-8e9e8d087072 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:09.760739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457b197-2068-4f97-97aa-8e9e8d087072 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:09.761003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457b197-2068-4f97-97aa-8e9e8d087072 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:09.770534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457b197-2068-4f97-97aa-8e9e8d087072 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:09.772428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457b197-2068-4f97-97aa-8e9e8d087072 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:09.800501 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1208/2400] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:24:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:24:10.388134 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bff41-b940-4359-bf10-ac6d514cb8e4 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:10.388698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bff41-b940-4359-bf10-ac6d514cb8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:10.388698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37bff41-b940-4359-bf10-ac6d514cb8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:10.419032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c37bff41-b940-4359-bf10-ac6d514cb8e4 None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:10.420580 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c37bff41-b940-4359-bf10-ac6d514cb8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:10.449662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1193/2401] 10.222.0.71 () {58 vars in 967 bytes} [Wed Sep 21 12:24:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:10.481556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457b197-2068-4f97-97aa-8e9e8d087072 service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:10.498714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457b197-2068-4f97-97aa-8e9e8d087072 service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:10.519323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2348af-29da-4820-8fad-a3da75db8d8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:10.520102 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2348af-29da-4820-8fad-a3da75db8d8d service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:10.520519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2348af-29da-4820-8fad-a3da75db8d8d service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:10.520864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2348af-29da-4820-8fad-a3da75db8d8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:10.538002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2348af-29da-4820-8fad-a3da75db8d8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:10.540639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2348af-29da-4820-8fad-a3da75db8d8d service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:10.567651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1209/2402] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:24:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:24:12.390097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f097995-4d62-4b88-ad4f-50fbb6bea1db None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:12.390603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f097995-4d62-4b88-ad4f-50fbb6bea1db None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:12.390603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f097995-4d62-4b88-ad4f-50fbb6bea1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:12.467423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9f097995-4d62-4b88-ad4f-50fbb6bea1db None None] MFA Rules not processed for user `8e0b6c8f5ec24d478f47a3d8d2fd941e`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:12.469311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9f097995-4d62-4b88-ad4f-50fbb6bea1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:12.612638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1194/2403] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:12.616982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2348af-29da-4820-8fad-a3da75db8d8d service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:12.632075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2348af-29da-4820-8fad-a3da75db8d8d service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:12.637466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99763fa7-fe69-4b0a-a9b3-78335f1cddd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:12.638588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99763fa7-fe69-4b0a-a9b3-78335f1cddd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:12.639083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99763fa7-fe69-4b0a-a9b3-78335f1cddd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:12.639545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99763fa7-fe69-4b0a-a9b3-78335f1cddd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:12.640426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99763fa7-fe69-4b0a-a9b3-78335f1cddd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:12.642725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99763fa7-fe69-4b0a-a9b3-78335f1cddd1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:12.678053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1210/2404] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:24:12 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:24:12.682536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f097995-4d62-4b88-ad4f-50fbb6bea1db None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:12.696914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f097995-4d62-4b88-ad4f-50fbb6bea1db None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:12.794303 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9a866-0684-4831-9f16-1a58a3ae362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:12.795410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9a866-0684-4831-9f16-1a58a3ae362b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:12.795883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9a866-0684-4831-9f16-1a58a3ae362b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:12.796506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9a866-0684-4831-9f16-1a58a3ae362b admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:12.797311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9a866-0684-4831-9f16-1a58a3ae362b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:12.800535 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9a866-0684-4831-9f16-1a58a3ae362b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:12.836144 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:12.851004 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1195/2405] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:24:12 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:24:12.854438 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99763fa7-fe69-4b0a-a9b3-78335f1cddd1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:12.867511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99763fa7-fe69-4b0a-a9b3-78335f1cddd1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:12.876852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c2ed3-d50e-4f25-bd4f-3d2f4303a3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:12.877443 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c2ed3-d50e-4f25-bd4f-3d2f4303a3ef admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:12.877791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c2ed3-d50e-4f25-bd4f-3d2f4303a3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:12.878099 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c2ed3-d50e-4f25-bd4f-3d2f4303a3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:12.879065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c2ed3-d50e-4f25-bd4f-3d2f4303a3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:12.882852 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c2ed3-d50e-4f25-bd4f-3d2f4303a3ef admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:12.898141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1211/2406] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:24:12.899886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9a866-0684-4831-9f16-1a58a3ae362b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:12.910881 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9a866-0684-4831-9f16-1a58a3ae362b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:12.916925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c07f26-41fc-4fb7-9454-23737a408d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:12.917853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c07f26-41fc-4fb7-9454-23737a408d24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:12.919125 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c07f26-41fc-4fb7-9454-23737a408d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:12.919497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c07f26-41fc-4fb7-9454-23737a408d24 admin admin] PATH_INFO: `/v3/projects/09c54af884ea427ba48e555db113973f/users/0be0164a7b834e75918b50ef771d2628/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:12.958165 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c07f26-41fc-4fb7-9454-23737a408d24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c54af884ea427ba48e555db113973f, user_id=0be0164a7b834e75918b50ef771d2628, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:12.961133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c07f26-41fc-4fb7-9454-23737a408d24 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:12.994320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1196/2407] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:24:12 2022] PUT /identity/v3/projects/09c54af884ea427ba48e555db113973f/users/0be0164a7b834e75918b50ef771d2628/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:12.999223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c2ed3-d50e-4f25-bd4f-3d2f4303a3ef admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:13.012584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c2ed3-d50e-4f25-bd4f-3d2f4303a3ef admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:13.151669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03784030-f396-4bda-a71b-ea616605baa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:13.152346 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03784030-f396-4bda-a71b-ea616605baa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:13.153986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03784030-f396-4bda-a71b-ea616605baa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:13.154276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03784030-f396-4bda-a71b-ea616605baa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:13.155005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03784030-f396-4bda-a71b-ea616605baa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:13.156767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03784030-f396-4bda-a71b-ea616605baa5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:13.158978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:13.176018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1212/2408] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:24:13.176386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c07f26-41fc-4fb7-9454-23737a408d24 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:13.211287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c07f26-41fc-4fb7-9454-23737a408d24 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:13.217372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48156e-25b5-41a7-a17c-181730e03446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:13.218059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48156e-25b5-41a7-a17c-181730e03446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:13.218344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48156e-25b5-41a7-a17c-181730e03446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:13.218702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48156e-25b5-41a7-a17c-181730e03446 admin admin] PATH_INFO: `/v3/projects/09c54af884ea427ba48e555db113973f/users/0be0164a7b834e75918b50ef771d2628/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:13.236034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48156e-25b5-41a7-a17c-181730e03446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09c54af884ea427ba48e555db113973f, user_id=0be0164a7b834e75918b50ef771d2628, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:13.238769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48156e-25b5-41a7-a17c-181730e03446 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:13.254459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1197/2409] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:24:13 2022] PUT /identity/v3/projects/09c54af884ea427ba48e555db113973f/users/0be0164a7b834e75918b50ef771d2628/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:24:13.607993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03784030-f396-4bda-a71b-ea616605baa5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:13.609885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-03784030-f396-4bda-a71b-ea616605baa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:13.631468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03784030-f396-4bda-a71b-ea616605baa5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:13.723629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:13.724629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:13.726034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:13.726485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] PATH_INFO: `/v3/users/0c80648c3936461295821ce1431a4164` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:13.730827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c80648c3936461295821ce1431a4164)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:13.735582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:13.815449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c80648c3936461295821ce1431a4164'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:13.832623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c80648c3936461295821ce1431a4164'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:13.833332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c80648c3936461295821ce1431a4164'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:13.834135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c80648c3936461295821ce1431a4164'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:13.842310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c80648c3936461295821ce1431a4164'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:13.865676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1213/2410] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:24:13 2022] DELETE /identity/v3/users/0c80648c3936461295821ce1431a4164 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:13.871080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b553a-c842-4b66-af39-34d895cf896a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:13.872423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b553a-c842-4b66-af39-34d895cf896a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:13.872423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b553a-c842-4b66-af39-34d895cf896a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:13.892406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-611b553a-c842-4b66-af39-34d895cf896a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:13.893627 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-611b553a-c842-4b66-af39-34d895cf896a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:13.992785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1198/2411] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:14.004277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:14.006036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:14.137472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ad28c-1d6f-40e7-b8ed-d172e0ec2e9d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:14.143235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88932e8b-8293-4068-8f82-574b9bc05ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:14.143896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88932e8b-8293-4068-8f82-574b9bc05ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:14.144461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88932e8b-8293-4068-8f82-574b9bc05ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:14.144990 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88932e8b-8293-4068-8f82-574b9bc05ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:14.157829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88932e8b-8293-4068-8f82-574b9bc05ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:14.159717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88932e8b-8293-4068-8f82-574b9bc05ed3 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:14.176467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1214/2412] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:24:14.209622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1365b7-7fbf-43c5-9cd8-f171c99eadcc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:14.209914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1365b7-7fbf-43c5-9cd8-f171c99eadcc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:14.210236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1365b7-7fbf-43c5-9cd8-f171c99eadcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:14.236880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-dc1365b7-7fbf-43c5-9cd8-f171c99eadcc None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:14.238432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dc1365b7-7fbf-43c5-9cd8-f171c99eadcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:14.314523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1199/2413] 10.222.0.71 () {58 vars in 967 bytes} [Wed Sep 21 12:24:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:14.329233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88932e8b-8293-4068-8f82-574b9bc05ed3 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:14.331304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-88932e8b-8293-4068-8f82-574b9bc05ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:14.370508 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:14.436493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88932e8b-8293-4068-8f82-574b9bc05ed3 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:14.442171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e96e4e-efc7-4f9b-94f2-308166d1c87f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:14.443432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e96e4e-efc7-4f9b-94f2-308166d1c87f service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:14.443432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e96e4e-efc7-4f9b-94f2-308166d1c87f service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:14.443700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e96e4e-efc7-4f9b-94f2-308166d1c87f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:14.453387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e96e4e-efc7-4f9b-94f2-308166d1c87f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:14.455437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e96e4e-efc7-4f9b-94f2-308166d1c87f service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:14.469203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1215/2414] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:24:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:24:14.687488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1365b7-7fbf-43c5-9cd8-f171c99eadcc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:14.689375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dc1365b7-7fbf-43c5-9cd8-f171c99eadcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:14.716480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1365b7-7fbf-43c5-9cd8-f171c99eadcc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:14.725499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:14.726505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:14.727583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:14.728187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] PATH_INFO: `/v3/projects/1a86f263882a44f385804b84f83d735c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:14.729748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a86f263882a44f385804b84f83d735c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:14.735064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:14.862747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a86f263882a44f385804b84f83d735c', 'request_id': 'req-47329ee6-13f9-449a-92b9-b4348eb9989f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:14.871748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a86f263882a44f385804b84f83d735c', 'request_id': 'req-47329ee6-13f9-449a-92b9-b4348eb9989f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:14.884002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a86f263882a44f385804b84f83d735c', 'request_id': 'req-47329ee6-13f9-449a-92b9-b4348eb9989f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:14.886756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:14.902383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] The token cache is being invalidate because project 1a86f263882a44f385804b84f83d735c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:24:14.903466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:14.904467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:14.906471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1200/2415] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:24:14 2022] DELETE /identity/v3/projects/1a86f263882a44f385804b84f83d735c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:17.092943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e96e4e-efc7-4f9b-94f2-308166d1c87f service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:17.097089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e9e96e4e-efc7-4f9b-94f2-308166d1c87f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:17.124329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e96e4e-efc7-4f9b-94f2-308166d1c87f service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:17.205189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:17.206018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:17.206306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:17.206587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] PATH_INFO: `/v3/users/1f8e995175ec4c6583e1ad0c7fb902ab` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:17.208150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f8e995175ec4c6583e1ad0c7fb902ab)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:17.210662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:17.309706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f8e995175ec4c6583e1ad0c7fb902ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:17.324302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f8e995175ec4c6583e1ad0c7fb902ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:17.325125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f8e995175ec4c6583e1ad0c7fb902ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:17.326232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f8e995175ec4c6583e1ad0c7fb902ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:17.337135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7949ef32-2658-4c43-9b66-f3cf8b4ddf85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f8e995175ec4c6583e1ad0c7fb902ab'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:17.385095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1216/2416] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:24:17 2022] DELETE /identity/v3/users/1f8e995175ec4c6583e1ad0c7fb902ab => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:17.935565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:17.937172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:17.954201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47329ee6-13f9-449a-92b9-b4348eb9989f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:18.064182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:18.065112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.065643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.066135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] PATH_INFO: `/v3/projects/c46922ef081642f6a3d8ad5d02e4155a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.068180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46922ef081642f6a3d8ad5d02e4155a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:18.071341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:18.204362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46922ef081642f6a3d8ad5d02e4155a', 'request_id': 'req-fdcc8b27-3e85-404e-b1bd-a162e3566384'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:18.212252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46922ef081642f6a3d8ad5d02e4155a', 'request_id': 'req-fdcc8b27-3e85-404e-b1bd-a162e3566384'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:18.225738 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46922ef081642f6a3d8ad5d02e4155a', 'request_id': 'req-fdcc8b27-3e85-404e-b1bd-a162e3566384'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:18.254849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] The token cache is being invalidate because project c46922ef081642f6a3d8ad5d02e4155a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:24:18.256616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:18.257407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fdcc8b27-3e85-404e-b1bd-a162e3566384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:18.259393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1201/2417] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:24:17 2022] DELETE /identity/v3/projects/c46922ef081642f6a3d8ad5d02e4155a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:18.264969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e90a8f-7a4a-41ae-b4d0-02db0ae335f9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.265921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e90a8f-7a4a-41ae-b4d0-02db0ae335f9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.266374 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e90a8f-7a4a-41ae-b4d0-02db0ae335f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.296856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f1e90a8f-7a4a-41ae-b4d0-02db0ae335f9 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:18.302436 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f1e90a8f-7a4a-41ae-b4d0-02db0ae335f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:18.341928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:18.407815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1217/2418] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:18.417121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6c714-99db-415b-a65f-5e59cb2c71a8 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.417428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6c714-99db-415b-a65f-5e59cb2c71a8 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.417716 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6c714-99db-415b-a65f-5e59cb2c71a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.460514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-88c6c714-99db-415b-a65f-5e59cb2c71a8 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:18.462548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-88c6c714-99db-415b-a65f-5e59cb2c71a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:18.477386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e90a8f-7a4a-41ae-b4d0-02db0ae335f9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:18.485174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1202/2419] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:18.490592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e90a8f-7a4a-41ae-b4d0-02db0ae335f9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:18.497361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034810c-dadd-49ae-b053-c641f90fa9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:18.498955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034810c-dadd-49ae-b053-c641f90fa9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.499653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034810c-dadd-49ae-b053-c641f90fa9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.499653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034810c-dadd-49ae-b053-c641f90fa9d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.500833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034810c-dadd-49ae-b053-c641f90fa9d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:24:18.501530 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034810c-dadd-49ae-b053-c641f90fa9d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:18.503053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034810c-dadd-49ae-b053-c641f90fa9d0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:18.532214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6c714-99db-415b-a65f-5e59cb2c71a8 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:18.539436 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1218/2420] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:24:18.554445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6c714-99db-415b-a65f-5e59cb2c71a8 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:18.581365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95edb0-25b5-4bae-b0b0-e1d90aefc221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:18.582417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95edb0-25b5-4bae-b0b0-e1d90aefc221 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.582851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95edb0-25b5-4bae-b0b0-e1d90aefc221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.583282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95edb0-25b5-4bae-b0b0-e1d90aefc221 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.584152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95edb0-25b5-4bae-b0b0-e1d90aefc221 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:18.586486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95edb0-25b5-4bae-b0b0-e1d90aefc221 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:18.623977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1203/2421] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:24:18 2022] POST /identity/v3/projects => generated 391 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:24:18.627910 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034810c-dadd-49ae-b053-c641f90fa9d0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:18.641036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034810c-dadd-49ae-b053-c641f90fa9d0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:18.724717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb4b4e-70bc-43e1-81e3-5f1cf4fc6ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:18.725864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb4b4e-70bc-43e1-81e3-5f1cf4fc6ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.726427 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb4b4e-70bc-43e1-81e3-5f1cf4fc6ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.726582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb4b4e-70bc-43e1-81e3-5f1cf4fc6ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.727611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb4b4e-70bc-43e1-81e3-5f1cf4fc6ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:18.729460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb4b4e-70bc-43e1-81e3-5f1cf4fc6ae8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:18.771474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1219/2422] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:24:18 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:24:18.776741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95edb0-25b5-4bae-b0b0-e1d90aefc221 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:18.788792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95edb0-25b5-4bae-b0b0-e1d90aefc221 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:18.794871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e651b-2581-4c7c-a631-6f221c6b465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:18.795514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e651b-2581-4c7c-a631-6f221c6b465d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.795820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e651b-2581-4c7c-a631-6f221c6b465d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.796107 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e651b-2581-4c7c-a631-6f221c6b465d admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.796776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e651b-2581-4c7c-a631-6f221c6b465d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:18.798821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e651b-2581-4c7c-a631-6f221c6b465d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:18.810562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1204/2423] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:24:18.814528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb4b4e-70bc-43e1-81e3-5f1cf4fc6ae8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:18.826818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb4b4e-70bc-43e1-81e3-5f1cf4fc6ae8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:18.832368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a93575-5696-4d1c-ab61-54717c01f2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:18.832978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a93575-5696-4d1c-ab61-54717c01f2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.833259 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a93575-5696-4d1c-ab61-54717c01f2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.833529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a93575-5696-4d1c-ab61-54717c01f2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.834217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a93575-5696-4d1c-ab61-54717c01f2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:18.835931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a93575-5696-4d1c-ab61-54717c01f2b8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:18.848150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1220/2424] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:24:18.852314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e651b-2581-4c7c-a631-6f221c6b465d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:18.865117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e651b-2581-4c7c-a631-6f221c6b465d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:18.870778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e55041-04fe-4459-bc91-710173764960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:18.871645 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e55041-04fe-4459-bc91-710173764960 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.871970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e55041-04fe-4459-bc91-710173764960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.872247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e55041-04fe-4459-bc91-710173764960 admin admin] PATH_INFO: `/v3/projects/fdc3af0678204f539662c81295cb15b8/users/1993771874a1496388e96a37987cdd3e/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.898520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e55041-04fe-4459-bc91-710173764960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc3af0678204f539662c81295cb15b8, user_id=1993771874a1496388e96a37987cdd3e, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:18.901644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e55041-04fe-4459-bc91-710173764960 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:18.923537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1205/2425] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:24:18 2022] PUT /identity/v3/projects/fdc3af0678204f539662c81295cb15b8/users/1993771874a1496388e96a37987cdd3e/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:18.934229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a8d50-4208-4e76-85a4-bf3f03b5c77b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:18.934513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a8d50-4208-4e76-85a4-bf3f03b5c77b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:18.934789 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a8d50-4208-4e76-85a4-bf3f03b5c77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:18.976795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c33a8d50-4208-4e76-85a4-bf3f03b5c77b None None] MFA Rules not processed for user `1993771874a1496388e96a37987cdd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:18.978583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c33a8d50-4208-4e76-85a4-bf3f03b5c77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:19.077266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1221/2426] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:19.091726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e55041-04fe-4459-bc91-710173764960 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:19.095678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e8e55041-04fe-4459-bc91-710173764960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:19.137756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bf3c0-8455-4590-8b4b-4272516898e4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:19.138034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bf3c0-8455-4590-8b4b-4272516898e4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:19.138308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bf3c0-8455-4590-8b4b-4272516898e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:19.139022 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:19.185780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-476bf3c0-8455-4590-8b4b-4272516898e4 None None] MFA Rules not processed for user `0be0164a7b834e75918b50ef771d2628`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:19.187179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-476bf3c0-8455-4590-8b4b-4272516898e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:19.200498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:19.203929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e55041-04fe-4459-bc91-710173764960 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:19.208293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1baae-50c7-47c2-8e57-bf89705f86a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:19.208902 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1baae-50c7-47c2-8e57-bf89705f86a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:19.209208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1baae-50c7-47c2-8e57-bf89705f86a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:19.209484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1baae-50c7-47c2-8e57-bf89705f86a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:19.219470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1baae-50c7-47c2-8e57-bf89705f86a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:19.220983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b1baae-50c7-47c2-8e57-bf89705f86a8 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:19.233342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1206/2427] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:24:19.271992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1222/2428] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:19.286064 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1baae-50c7-47c2-8e57-bf89705f86a8 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:19.288195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-51b1baae-50c7-47c2-8e57-bf89705f86a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:19.348788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1baae-50c7-47c2-8e57-bf89705f86a8 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:19.353504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56a94ccca15c462c9e7a5e88b907f420', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:19.354124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:19.354408 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:19.354683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:19.356955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:19.449057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:19.451133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:19.466653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1207/2429] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:24:19.774925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dda95-53eb-4af4-9954-c4783c55356f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:19.775241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dda95-53eb-4af4-9954-c4783c55356f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:19.775484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dda95-53eb-4af4-9954-c4783c55356f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:19.802956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-075dda95-53eb-4af4-9954-c4783c55356f None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:19.804314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-075dda95-53eb-4af4-9954-c4783c55356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:19.890162 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1223/2430] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:19.906516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:19.908214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:20.002252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e2039-274a-4bbb-b48d-9570d506b9da service glance] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:20.038460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a615854-8860-4360-b6a1-af15d9c649ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:20.039124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a615854-8860-4360-b6a1-af15d9c649ac service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:20.039407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a615854-8860-4360-b6a1-af15d9c649ac service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:20.039680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a615854-8860-4360-b6a1-af15d9c649ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:20.049113 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a615854-8860-4360-b6a1-af15d9c649ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:20.050709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a615854-8860-4360-b6a1-af15d9c649ac service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:20.060020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1208/2431] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:24:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 12:24:29.437958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7c8b1-0a69-4b8e-b3ca-b710bb4dfe3b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:29.438467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7c8b1-0a69-4b8e-b3ca-b710bb4dfe3b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:29.438558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7c8b1-0a69-4b8e-b3ca-b710bb4dfe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:29.457679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-62a7c8b1-0a69-4b8e-b3ca-b710bb4dfe3b None None] MFA Rules not processed for user `8e0b6c8f5ec24d478f47a3d8d2fd941e`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:29.458919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-62a7c8b1-0a69-4b8e-b3ca-b710bb4dfe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:29.470974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1224/2432] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:30.055938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:31.821840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53351a55-64da-4270-9b15-a947f23b462b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:31.822460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53351a55-64da-4270-9b15-a947f23b462b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:31.822460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53351a55-64da-4270-9b15-a947f23b462b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:31.842053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-53351a55-64da-4270-9b15-a947f23b462b None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:31.843305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-53351a55-64da-4270-9b15-a947f23b462b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:31.850665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1209/2433] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:31.864550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7c8b1-0a69-4b8e-b3ca-b710bb4dfe3b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:31.878274 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7c8b1-0a69-4b8e-b3ca-b710bb4dfe3b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:31.883481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c08ff9-a798-4cb9-a82a-47aff410d8f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:31.884419 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08ff9-a798-4cb9-a82a-47aff410d8f8 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:31.885005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08ff9-a798-4cb9-a82a-47aff410d8f8 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:31.885350 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08ff9-a798-4cb9-a82a-47aff410d8f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:31.894979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c08ff9-a798-4cb9-a82a-47aff410d8f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:31.896761 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c08ff9-a798-4cb9-a82a-47aff410d8f8 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:31.906045 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1225/2434] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:24:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:24:41.836156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:41.901947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=58 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:45.318393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53351a55-64da-4270-9b15-a947f23b462b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:45.390036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53351a55-64da-4270-9b15-a947f23b462b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:45.394715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:45.395370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:45.395646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:45.396080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] PATH_INFO: `/v3/users/1993771874a1496388e96a37987cdd3e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:45.397216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1993771874a1496388e96a37987cdd3e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:45.399163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:45.452124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1993771874a1496388e96a37987cdd3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:45.452869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1993771874a1496388e96a37987cdd3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:45.461209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1993771874a1496388e96a37987cdd3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:45.462213 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1993771874a1496388e96a37987cdd3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:45.470188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1993771874a1496388e96a37987cdd3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:45.490495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1210/2435] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:24:45 2022] DELETE /identity/v3/users/1993771874a1496388e96a37987cdd3e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:45.495233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f2cba-bd6c-42d6-874b-88718ce06d31 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:45.495829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f2cba-bd6c-42d6-874b-88718ce06d31 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:45.495829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f2cba-bd6c-42d6-874b-88718ce06d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:45.517611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-540f2cba-bd6c-42d6-874b-88718ce06d31 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:24:45.518825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-540f2cba-bd6c-42d6-874b-88718ce06d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:45.580181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1226/2436] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:24:45.590179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:45.591625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:45.662996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e97b63-6f71-436a-96c4-3f29b0ad3284 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:45.666978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d879c99-6968-44ec-9821-628a734a33de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:45.668050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879c99-6968-44ec-9821-628a734a33de service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:45.668487 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879c99-6968-44ec-9821-628a734a33de service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:45.668929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879c99-6968-44ec-9821-628a734a33de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:45.680067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d879c99-6968-44ec-9821-628a734a33de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:45.681924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d879c99-6968-44ec-9821-628a734a33de service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:45.693960 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1211/2437] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:24:45.974307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f2cba-bd6c-42d6-874b-88718ce06d31 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:24:45.975773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-540f2cba-bd6c-42d6-874b-88718ce06d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:24:45.990191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f2cba-bd6c-42d6-874b-88718ce06d31 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:24:45.994963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:24:45.995719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:24:45.996122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:24:45.996472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] PATH_INFO: `/v3/projects/fdc3af0678204f539662c81295cb15b8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:24:45.997701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc3af0678204f539662c81295cb15b8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:24:45.999641 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:24:46.057151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc3af0678204f539662c81295cb15b8', 'request_id': 'req-334c88d3-0a03-4869-aa83-0eea0dcc3791'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:46.063273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc3af0678204f539662c81295cb15b8', 'request_id': 'req-334c88d3-0a03-4869-aa83-0eea0dcc3791'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:46.070976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc3af0678204f539662c81295cb15b8', 'request_id': 'req-334c88d3-0a03-4869-aa83-0eea0dcc3791'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:46.082671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] The token cache is being invalidate because project fdc3af0678204f539662c81295cb15b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:24:46.082959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:46.083670 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-334c88d3-0a03-4869-aa83-0eea0dcc3791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:24:46.085256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1227/2438] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:24:45 2022] DELETE /identity/v3/projects/fdc3af0678204f539662c81295cb15b8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:24:55.688904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:24:56.079345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:25:03.907618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc791b4c-4ac9-4edf-8018-67ae05418f6a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:03.908145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc791b4c-4ac9-4edf-8018-67ae05418f6a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:03.908269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc791b4c-4ac9-4edf-8018-67ae05418f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:03.930259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-bc791b4c-4ac9-4edf-8018-67ae05418f6a None None] MFA Rules not processed for user `0fb9105e474a4a29904722b1e33d0155`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:25:03.931462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bc791b4c-4ac9-4edf-8018-67ae05418f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:03.986443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1212/2439] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:25:08.799976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1a556-74b0-4f76-a4c1-d6d78256643a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:08.801036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1a556-74b0-4f76-a4c1-d6d78256643a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:08.801494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1a556-74b0-4f76-a4c1-d6d78256643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:08.823157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-66f1a556-74b0-4f76-a4c1-d6d78256643a None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:25:08.832921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-66f1a556-74b0-4f76-a4c1-d6d78256643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:08.901187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1228/2440] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:25:08.922067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc791b4c-4ac9-4edf-8018-67ae05418f6a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:25:08.923545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bc791b4c-4ac9-4edf-8018-67ae05418f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:09.001387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc791b4c-4ac9-4edf-8018-67ae05418f6a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:25:09.006354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6ac79-6ef9-46dd-a154-51c4d48c54ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:25:09.006960 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6ac79-6ef9-46dd-a154-51c4d48c54ab service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:09.007241 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6ac79-6ef9-46dd-a154-51c4d48c54ab service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:09.007563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6ac79-6ef9-46dd-a154-51c4d48c54ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:09.016190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6ac79-6ef9-46dd-a154-51c4d48c54ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:25:09.017760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6ac79-6ef9-46dd-a154-51c4d48c54ab service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:25:09.025652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1213/2441] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:25:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:25:18.891515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=25 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:25:19.023082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=57 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:25:20.128260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1a556-74b0-4f76-a4c1-d6d78256643a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:25:20.130796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-66f1a556-74b0-4f76-a4c1-d6d78256643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:20.196071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1a556-74b0-4f76-a4c1-d6d78256643a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:25:20.199844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d706d-d6d1-4274-bc60-98e1ec0e4e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:25:20.200454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d706d-d6d1-4274-bc60-98e1ec0e4e61 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:20.200731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d706d-d6d1-4274-bc60-98e1ec0e4e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:20.201003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d706d-d6d1-4274-bc60-98e1ec0e4e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:20.203989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-127d706d-d6d1-4274-bc60-98e1ec0e4e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:20.270097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d706d-d6d1-4274-bc60-98e1ec0e4e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:25:20.271674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d706d-d6d1-4274-bc60-98e1ec0e4e61 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:25:20.281829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1229/2442] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:25:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:25:26.369781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6dbc5-22fa-45a9-94b9-18c05ded6394 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:26.370758 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6dbc5-22fa-45a9-94b9-18c05ded6394 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:26.371322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6dbc5-22fa-45a9-94b9-18c05ded6394 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:26.373008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1214/2443] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:25:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:25:26.379274 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b16b87-d991-438e-8eef-059bf1668e5a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:26.379838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b16b87-d991-438e-8eef-059bf1668e5a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:26.379947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b16b87-d991-438e-8eef-059bf1668e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:26.401987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-86b16b87-d991-438e-8eef-059bf1668e5a None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:25:26.403206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-86b16b87-d991-438e-8eef-059bf1668e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:26.487146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1230/2444] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:25:26.500027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6dbc5-22fa-45a9-94b9-18c05ded6394 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:25:26.501744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b1e6dbc5-22fa-45a9-94b9-18c05ded6394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:26.563164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6dbc5-22fa-45a9-94b9-18c05ded6394 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:25:26.566785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e143b9-7d7c-4f10-bd00-16f0667325fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56a94ccca15c462c9e7a5e88b907f420', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:25:26.567406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e143b9-7d7c-4f10-bd00-16f0667325fe service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:26.567825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e143b9-7d7c-4f10-bd00-16f0667325fe service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:26.568126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e143b9-7d7c-4f10-bd00-16f0667325fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:26.579908 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e143b9-7d7c-4f10-bd00-16f0667325fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:25:26.581348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e143b9-7d7c-4f10-bd00-16f0667325fe service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:25:26.593341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1215/2445] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:25:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:25:26.662586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821e338-ba6a-4a26-9e59-5c39f4cd3fc9 None None] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:26.662881 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821e338-ba6a-4a26-9e59-5c39f4cd3fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:26.663156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821e338-ba6a-4a26-9e59-5c39f4cd3fc9 None None] PATH_INFO: `/` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:26.664557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1231/2446] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:25:26.669272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d33a81-7902-4460-9a9b-44d74068347b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:26.669556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d33a81-7902-4460-9a9b-44d74068347b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:26.669845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d33a81-7902-4460-9a9b-44d74068347b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:26.689233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-16d33a81-7902-4460-9a9b-44d74068347b None None] MFA Rules not processed for user `cd66d5d9989449729cec7e60cdaf0949`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:25:26.690448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-16d33a81-7902-4460-9a9b-44d74068347b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:26.696943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1216/2447] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:25:26.709748 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821e338-ba6a-4a26-9e59-5c39f4cd3fc9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:25:26.711473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c821e338-ba6a-4a26-9e59-5c39f4cd3fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:26.725218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c821e338-ba6a-4a26-9e59-5c39f4cd3fc9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:25:26.728805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfbb15-5f45-46b2-b46c-e4f49b515b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:25:26.729401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfbb15-5f45-46b2-b46c-e4f49b515b17 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:26.729681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfbb15-5f45-46b2-b46c-e4f49b515b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:26.729953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfbb15-5f45-46b2-b46c-e4f49b515b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:26.737552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfbb15-5f45-46b2-b46c-e4f49b515b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:25:26.738977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bfbb15-5f45-46b2-b46c-e4f49b515b17 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:25:26.746814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1232/2448] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:25:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:25:26.830905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6abbb6-d3f5-4db3-97ff-0401a14a71f5 None None] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:26.831188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6abbb6-d3f5-4db3-97ff-0401a14a71f5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:26.831465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6abbb6-d3f5-4db3-97ff-0401a14a71f5 None None] PATH_INFO: `/` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:26.832977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1217/2449] 10.222.0.71 () {54 vars in 921 bytes} [Wed Sep 21 12:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 12:25:26.839324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2827b-b7ff-4b02-bb81-33a28a8182be None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:26.839612 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2827b-b7ff-4b02-bb81-33a28a8182be None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:26.839919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2827b-b7ff-4b02-bb81-33a28a8182be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:26.860656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-77e2827b-b7ff-4b02-bb81-33a28a8182be None None] MFA Rules not processed for user `463487449c1d4aada1b5f279c959b0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:25:26.861788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-77e2827b-b7ff-4b02-bb81-33a28a8182be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:26.871202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1233/2450] 10.222.0.71 () {58 vars in 1018 bytes} [Wed Sep 21 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:25:26.884216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6abbb6-d3f5-4db3-97ff-0401a14a71f5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:25:26.895295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6abbb6-d3f5-4db3-97ff-0401a14a71f5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:25:26.898868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c04a0-6095-4eb9-b589-ec7765dde3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:25:26.899461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c04a0-6095-4eb9-b589-ec7765dde3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:26.899737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c04a0-6095-4eb9-b589-ec7765dde3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:26.900031 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c04a0-6095-4eb9-b589-ec7765dde3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:26.910460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c04a0-6095-4eb9-b589-ec7765dde3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:25:26.911935 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c04a0-6095-4eb9-b589-ec7765dde3c9 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:25:26.922956 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1218/2451] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:25:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:25:29.902079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18a265-d0ab-442a-831b-58b774998680 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:29.902500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18a265-d0ab-442a-831b-58b774998680 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:29.902683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18a265-d0ab-442a-831b-58b774998680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:29.922961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bc18a265-d0ab-442a-831b-58b774998680 None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:25:29.924281 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bc18a265-d0ab-442a-831b-58b774998680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:29.936798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1234/2452] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:25:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:25:29.949436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c04a0-6095-4eb9-b589-ec7765dde3c9 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:25:29.961200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c04a0-6095-4eb9-b589-ec7765dde3c9 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:25:29.966423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e1ac5-1835-4a0e-864c-1703b5624ccb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:25:29.967515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e1ac5-1835-4a0e-864c-1703b5624ccb service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:29.967927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e1ac5-1835-4a0e-864c-1703b5624ccb service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:29.968282 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e1ac5-1835-4a0e-864c-1703b5624ccb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:29.978083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e1ac5-1835-4a0e-864c-1703b5624ccb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:25:29.979780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e1ac5-1835-4a0e-864c-1703b5624ccb service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:25:29.988741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1219/2453] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:25:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:25:39.439147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18a265-d0ab-442a-831b-58b774998680 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:25:39.448639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18a265-d0ab-442a-831b-58b774998680 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:25:39.452292 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:25:39.453152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:39.453426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:39.453693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:39.455449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:39.469642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:25:39.471071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:25:39.483477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1235/2454] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:25:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:25:39.985571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=41 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:25:40.687875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e1ac5-1835-4a0e-864c-1703b5624ccb service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:25:40.697682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e1ac5-1835-4a0e-864c-1703b5624ccb service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:25:40.701744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6509b8-4c90-4c6d-a4bc-19f74636a12c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:25:40.702447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6509b8-4c90-4c6d-a4bc-19f74636a12c service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:40.702799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6509b8-4c90-4c6d-a4bc-19f74636a12c service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:40.703396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6509b8-4c90-4c6d-a4bc-19f74636a12c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:40.705372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ef6509b8-4c90-4c6d-a4bc-19f74636a12c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:40.720135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6509b8-4c90-4c6d-a4bc-19f74636a12c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:25:40.721767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6509b8-4c90-4c6d-a4bc-19f74636a12c service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:25:40.733778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1220/2455] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:25:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:25:49.477276 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:25:50.728318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:25:51.308227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:25:51.309648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:51.324293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fde7b8-2419-4d69-86d7-380d0993303a service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:25:51.330220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d406e80-7c3b-4fac-84bf-5241e9ccf2cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:25:51.331011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d406e80-7c3b-4fac-84bf-5241e9ccf2cb service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:25:51.331294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d406e80-7c3b-4fac-84bf-5241e9ccf2cb service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:25:51.331563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d406e80-7c3b-4fac-84bf-5241e9ccf2cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:25:51.333287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7d406e80-7c3b-4fac-84bf-5241e9ccf2cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:25:51.390893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d406e80-7c3b-4fac-84bf-5241e9ccf2cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:25:51.392545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d406e80-7c3b-4fac-84bf-5241e9ccf2cb service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:25:51.401606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1236/2456] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:25:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:26:00.215802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2075f8-c385-442a-9e73-01d4d35752b5 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:26:00.216305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2075f8-c385-442a-9e73-01d4d35752b5 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:26:00.216495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2075f8-c385-442a-9e73-01d4d35752b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:26:00.236135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fe2075f8-c385-442a-9e73-01d4d35752b5 None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:26:00.237355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fe2075f8-c385-442a-9e73-01d4d35752b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:26:00.244766 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1221/2457] 10.222.0.71 () {58 vars in 967 bytes} [Wed Sep 21 12:26:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:26:00.257978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d406e80-7c3b-4fac-84bf-5241e9ccf2cb service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:26:00.269206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d406e80-7c3b-4fac-84bf-5241e9ccf2cb service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:26:00.274177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db8498-3b98-4464-a5a4-1b4672595ffa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:26:00.274779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db8498-3b98-4464-a5a4-1b4672595ffa service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:26:00.275056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db8498-3b98-4464-a5a4-1b4672595ffa service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:26:00.275330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db8498-3b98-4464-a5a4-1b4672595ffa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:26:00.283369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db8498-3b98-4464-a5a4-1b4672595ffa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:26:00.285012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db8498-3b98-4464-a5a4-1b4672595ffa service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:26:00.293042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1237/2458] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:26:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:26:10.230612 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:26:10.289738 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=34 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:26:43.369201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2075f8-c385-442a-9e73-01d4d35752b5 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:26:43.382051 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2075f8-c385-442a-9e73-01d4d35752b5 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:26:43.387356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7ddaf-14dd-429e-8d78-93ff5ec9ad35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:26:43.387997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7ddaf-14dd-429e-8d78-93ff5ec9ad35 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:26:43.388276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7ddaf-14dd-429e-8d78-93ff5ec9ad35 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:26:43.388550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7ddaf-14dd-429e-8d78-93ff5ec9ad35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:26:43.390261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8ad7ddaf-14dd-429e-8d78-93ff5ec9ad35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:26:43.447563 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7ddaf-14dd-429e-8d78-93ff5ec9ad35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:26:43.449168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7ddaf-14dd-429e-8d78-93ff5ec9ad35 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:26:43.458452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1222/2459] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:26:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:26:48.594009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db8498-3b98-4464-a5a4-1b4672595ffa service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:26:48.605314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db8498-3b98-4464-a5a4-1b4672595ffa service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:26:48.609936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57f106-7834-4b3d-a8be-bff76a9dfe12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:26:48.611009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f106-7834-4b3d-a8be-bff76a9dfe12 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:26:48.611512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f106-7834-4b3d-a8be-bff76a9dfe12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:26:48.612032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f106-7834-4b3d-a8be-bff76a9dfe12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:26:48.614301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fb57f106-7834-4b3d-a8be-bff76a9dfe12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:26:48.640267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57f106-7834-4b3d-a8be-bff76a9dfe12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:26:48.640267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57f106-7834-4b3d-a8be-bff76a9dfe12 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:26:48.640267 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1238/2460] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:26:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:26:53.454272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=28 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:26:58.634774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:27:16.642019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7ddaf-14dd-429e-8d78-93ff5ec9ad35 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:27:16.654679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7ddaf-14dd-429e-8d78-93ff5ec9ad35 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:27:16.662911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00aaa19-fd26-4314-b5a8-147558807538 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:27:16.664034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00aaa19-fd26-4314-b5a8-147558807538 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:27:16.664577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00aaa19-fd26-4314-b5a8-147558807538 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:27:16.665102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00aaa19-fd26-4314-b5a8-147558807538 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:27:16.667403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e00aaa19-fd26-4314-b5a8-147558807538 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:27:16.681664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00aaa19-fd26-4314-b5a8-147558807538 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:27:16.683381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00aaa19-fd26-4314-b5a8-147558807538 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:27:16.695152 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1223/2461] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:27:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:27:21.274013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64002a56-7693-42b6-a74d-7768f38d7ec2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:27:21.274526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64002a56-7693-42b6-a74d-7768f38d7ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:27:21.274642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64002a56-7693-42b6-a74d-7768f38d7ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:27:21.294669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-64002a56-7693-42b6-a74d-7768f38d7ec2 None None] MFA Rules not processed for user `c950eb00b9474b29b86d24983e3b1583`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:27:21.295950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64002a56-7693-42b6-a74d-7768f38d7ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:27:21.302776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1239/2462] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:27:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:27:21.611453 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00aaa19-fd26-4314-b5a8-147558807538 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:27:21.622195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00aaa19-fd26-4314-b5a8-147558807538 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:27:21.626425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7412b2c-bc0b-4342-9f23-200a33611e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:27:21.627032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7412b2c-bc0b-4342-9f23-200a33611e83 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:27:21.627320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7412b2c-bc0b-4342-9f23-200a33611e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:27:21.627625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7412b2c-bc0b-4342-9f23-200a33611e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:27:21.629383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a7412b2c-bc0b-4342-9f23-200a33611e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:27:21.639438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7412b2c-bc0b-4342-9f23-200a33611e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:27:21.642108 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7412b2c-bc0b-4342-9f23-200a33611e83 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:27:21.650788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1224/2463] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:27:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:27:21.970646 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64002a56-7693-42b6-a74d-7768f38d7ec2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:27:21.972174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-64002a56-7693-42b6-a74d-7768f38d7ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:27:22.042029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64002a56-7693-42b6-a74d-7768f38d7ec2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:27:22.047523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:27:22.048157 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:27:22.048440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:27:22.048705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:27:22.050446 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:27:22.061534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:27:22.063647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:27:22.073266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1240/2464] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:27:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:27:27.727104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bc82c-24a0-4498-89fd-d1b77ecaaf46 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:27:27.727607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bc82c-24a0-4498-89fd-d1b77ecaaf46 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:27:27.727705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bc82c-24a0-4498-89fd-d1b77ecaaf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:27:27.755057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e56bc82c-24a0-4498-89fd-d1b77ecaaf46 None None] MFA Rules not processed for user `0fb9105e474a4a29904722b1e33d0155`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:27:27.756321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e56bc82c-24a0-4498-89fd-d1b77ecaaf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:27:27.764419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1225/2465] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:27:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:27:32.069199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=35 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:27:33.613781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:27:33.615312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:27:33.641962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c4d26-8a33-4df7-8ace-9efdcd346df1 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:27:33.650753 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820be208-ba77-4eee-a82b-90cef5488142 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:27:33.651409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820be208-ba77-4eee-a82b-90cef5488142 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:27:33.651706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820be208-ba77-4eee-a82b-90cef5488142 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:27:33.652234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820be208-ba77-4eee-a82b-90cef5488142 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:27:33.654098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-820be208-ba77-4eee-a82b-90cef5488142 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:27:33.669649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820be208-ba77-4eee-a82b-90cef5488142 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:27:33.671293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820be208-ba77-4eee-a82b-90cef5488142 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:27:33.682962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1241/2466] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:27:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:27:35.238287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bc82c-24a0-4498-89fd-d1b77ecaaf46 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:27:35.255338 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bc82c-24a0-4498-89fd-d1b77ecaaf46 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:27:35.260912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86131f03-f0b0-4dd8-a1bf-dced6da747d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:27:35.261530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86131f03-f0b0-4dd8-a1bf-dced6da747d4 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:27:35.261810 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86131f03-f0b0-4dd8-a1bf-dced6da747d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:27:35.262087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86131f03-f0b0-4dd8-a1bf-dced6da747d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:27:35.263861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-86131f03-f0b0-4dd8-a1bf-dced6da747d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:27:35.277386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86131f03-f0b0-4dd8-a1bf-dced6da747d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:27:35.278895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86131f03-f0b0-4dd8-a1bf-dced6da747d4 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:27:35.290685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1226/2467] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:27:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:27:43.678382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:27:45.285884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=21 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:28:09.050921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75d247-612d-4b21-988f-f3da45b2febf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:09.051452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75d247-612d-4b21-988f-f3da45b2febf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:09.051452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75d247-612d-4b21-988f-f3da45b2febf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:09.071864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ac75d247-612d-4b21-988f-f3da45b2febf None None] MFA Rules not processed for user `8790c086dcaa4da5b739bcd02d30b4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:28:09.073091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ac75d247-612d-4b21-988f-f3da45b2febf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:09.080970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1242/2468] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:28:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:28:16.091059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b393a47-4af2-4963-8464-2b2c25dc9439 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:16.092098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b393a47-4af2-4963-8464-2b2c25dc9439 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:16.092191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b393a47-4af2-4963-8464-2b2c25dc9439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:16.111580 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0b393a47-4af2-4963-8464-2b2c25dc9439 None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:28:16.112802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b393a47-4af2-4963-8464-2b2c25dc9439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:16.120024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1227/2469] 10.222.0.71 () {58 vars in 967 bytes} [Wed Sep 21 12:28:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:28:16.165729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75d247-612d-4b21-988f-f3da45b2febf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:16.178220 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75d247-612d-4b21-988f-f3da45b2febf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:16.183171 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c80a88-d5b7-47ae-8be9-da30d7e555a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:16.183792 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c80a88-d5b7-47ae-8be9-da30d7e555a5 service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:16.184080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c80a88-d5b7-47ae-8be9-da30d7e555a5 service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:16.184355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c80a88-d5b7-47ae-8be9-da30d7e555a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:16.192238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c80a88-d5b7-47ae-8be9-da30d7e555a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:16.193853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c80a88-d5b7-47ae-8be9-da30d7e555a5 service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:16.202017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1243/2470] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:28:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:28:20.296892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b393a47-4af2-4963-8464-2b2c25dc9439 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:20.311431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b393a47-4af2-4963-8464-2b2c25dc9439 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:20.317471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:20.318181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:20.318521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:20.318865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:20.321644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:20.336451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:20.338116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:20.349875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1228/2471] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:28:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:28:23.988407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c80a88-d5b7-47ae-8be9-da30d7e555a5 service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:24.001140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c80a88-d5b7-47ae-8be9-da30d7e555a5 service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:24.006239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4de2-c23e-4eef-8085-ac7560aa32b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:24.006772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4de2-c23e-4eef-8085-ac7560aa32b2 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:24.007053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4de2-c23e-4eef-8085-ac7560aa32b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:24.007324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4de2-c23e-4eef-8085-ac7560aa32b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:24.009063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-de0e4de2-c23e-4eef-8085-ac7560aa32b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:24.021617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4de2-c23e-4eef-8085-ac7560aa32b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:24.023511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4de2-c23e-4eef-8085-ac7560aa32b2 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:24.037142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1244/2472] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:28:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:28:24.045553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:24.046846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:24.115005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faeaafb-a0f4-4d6d-b9b3-de71c520f7c8 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:24.119228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:24.119896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:24.120181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:24.120458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] PATH_INFO: `/v3/users/8e0b6c8f5ec24d478f47a3d8d2fd941e` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:24.121577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e0b6c8f5ec24d478f47a3d8d2fd941e)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:24.124180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:24.170384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e0b6c8f5ec24d478f47a3d8d2fd941e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.171023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e0b6c8f5ec24d478f47a3d8d2fd941e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.179181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e0b6c8f5ec24d478f47a3d8d2fd941e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.179845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e0b6c8f5ec24d478f47a3d8d2fd941e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.187850 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e0b6c8f5ec24d478f47a3d8d2fd941e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.205067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1229/2473] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:28:24 2022] DELETE /identity/v3/users/8e0b6c8f5ec24d478f47a3d8d2fd941e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:28:24.376844 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4de2-c23e-4eef-8085-ac7560aa32b2 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:24.389571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4de2-c23e-4eef-8085-ac7560aa32b2 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:24.463824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:24.464460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:24.464745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:24.465016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] PATH_INFO: `/v3/projects/9de710fda38c429a91595545e9a74dcb` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:24.466644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9de710fda38c429a91595545e9a74dcb)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:24.468617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:24.514526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9de710fda38c429a91595545e9a74dcb', 'request_id': 'req-6fb27538-0638-40bf-a48e-053669de789c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.520456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9de710fda38c429a91595545e9a74dcb', 'request_id': 'req-6fb27538-0638-40bf-a48e-053669de789c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.528924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9de710fda38c429a91595545e9a74dcb', 'request_id': 'req-6fb27538-0638-40bf-a48e-053669de789c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.539552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] The token cache is being invalidate because project 9de710fda38c429a91595545e9a74dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:28:24.539912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.540529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.542047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1245/2474] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:28:24 2022] DELETE /identity/v3/projects/9de710fda38c429a91595545e9a74dcb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:28:24.546717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:24.548185 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:24.561098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b9b5-23b8-46e9-b74d-fae9ee79ce4b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:24.619650 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:24.620312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:24.620593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:24.620872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] PATH_INFO: `/v3/users/0be0164a7b834e75918b50ef771d2628` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:24.622000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be0164a7b834e75918b50ef771d2628)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:24.623984 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:24.625473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:28:24.680730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be0164a7b834e75918b50ef771d2628'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.681401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be0164a7b834e75918b50ef771d2628'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.689979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be0164a7b834e75918b50ef771d2628'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.690591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be0164a7b834e75918b50ef771d2628'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.698438 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-06802c71-d4db-4faa-ab5d-f3454b121164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be0164a7b834e75918b50ef771d2628'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:24.717752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1230/2475] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:28:24 2022] DELETE /identity/v3/users/0be0164a7b834e75918b50ef771d2628 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:28:24.933839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:24.935983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:24.963609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb27538-0638-40bf-a48e-053669de789c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:25.025870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:28:25.033518 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:25.034584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.035053 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.035327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] PATH_INFO: `/v3/projects/09c54af884ea427ba48e555db113973f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.036724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09c54af884ea427ba48e555db113973f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:25.039068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:25.085743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09c54af884ea427ba48e555db113973f', 'request_id': 'req-114c526c-b3fa-4526-8347-5b9ba1aa46b3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:25.091973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09c54af884ea427ba48e555db113973f', 'request_id': 'req-114c526c-b3fa-4526-8347-5b9ba1aa46b3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:25.100315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09c54af884ea427ba48e555db113973f', 'request_id': 'req-114c526c-b3fa-4526-8347-5b9ba1aa46b3'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:25.111318 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] The token cache is being invalidate because project 09c54af884ea427ba48e555db113973f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:28:25.111599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:25.112294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-114c526c-b3fa-4526-8347-5b9ba1aa46b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:28:25.114217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1246/2476] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:28:24 2022] DELETE /identity/v3/projects/09c54af884ea427ba48e555db113973f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:28:25.118820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25638121-05df-4e1b-8f92-caebb7339b22 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.119327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25638121-05df-4e1b-8f92-caebb7339b22 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.119431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25638121-05df-4e1b-8f92-caebb7339b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.140115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-25638121-05df-4e1b-8f92-caebb7339b22 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:28:25.141358 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-25638121-05df-4e1b-8f92-caebb7339b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:25.205114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1231/2477] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:28:25.213396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd7ef8-7968-44bd-ba6b-15030c2df07c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.213743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd7ef8-7968-44bd-ba6b-15030c2df07c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.214024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd7ef8-7968-44bd-ba6b-15030c2df07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.234576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4bbd7ef8-7968-44bd-ba6b-15030c2df07c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:28:25.235818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4bbd7ef8-7968-44bd-ba6b-15030c2df07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:25.243947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1247/2478] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:28:25.248739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25638121-05df-4e1b-8f92-caebb7339b22 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:25.258265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25638121-05df-4e1b-8f92-caebb7339b22 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:25.263392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7165e50-f0e8-4b98-904f-ad699b56ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:25.264028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7165e50-f0e8-4b98-904f-ad699b56ab28 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.264309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7165e50-f0e8-4b98-904f-ad699b56ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.264577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7165e50-f0e8-4b98-904f-ad699b56ab28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.265220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7165e50-f0e8-4b98-904f-ad699b56ab28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:28:25.265519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7165e50-f0e8-4b98-904f-ad699b56ab28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:25.267117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7165e50-f0e8-4b98-904f-ad699b56ab28 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:25.279118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1232/2479] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:28:25.283473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd7ef8-7968-44bd-ba6b-15030c2df07c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:25.293624 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd7ef8-7968-44bd-ba6b-15030c2df07c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:25.302699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed761535-2ce3-4d52-9f69-45f653690f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:25.304398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761535-2ce3-4d52-9f69-45f653690f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.304398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761535-2ce3-4d52-9f69-45f653690f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.304398 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed761535-2ce3-4d52-9f69-45f653690f65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.304699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed761535-2ce3-4d52-9f69-45f653690f65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:25.306510 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed761535-2ce3-4d52-9f69-45f653690f65 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:25.342697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1248/2480] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:28:25 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:28:25.346406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7165e50-f0e8-4b98-904f-ad699b56ab28 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:25.356520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7165e50-f0e8-4b98-904f-ad699b56ab28 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:25.419551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1cd63-d8d1-4d12-9e43-8743edf3bbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:25.420507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1cd63-d8d1-4d12-9e43-8743edf3bbfa admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.420507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1cd63-d8d1-4d12-9e43-8743edf3bbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.420773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1cd63-d8d1-4d12-9e43-8743edf3bbfa admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.421570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1cd63-d8d1-4d12-9e43-8743edf3bbfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:25.423322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1cd63-d8d1-4d12-9e43-8743edf3bbfa admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:25.435597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:28:25.450774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1233/2481] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:28:25 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:28:25.452010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed761535-2ce3-4d52-9f69-45f653690f65 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:25.462756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed761535-2ce3-4d52-9f69-45f653690f65 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:25.467492 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea04b7-640a-420e-b1d0-e6b87be1779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:25.468150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea04b7-640a-420e-b1d0-e6b87be1779b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.468458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea04b7-640a-420e-b1d0-e6b87be1779b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.468732 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea04b7-640a-420e-b1d0-e6b87be1779b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.469373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea04b7-640a-420e-b1d0-e6b87be1779b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:25.470980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea04b7-640a-420e-b1d0-e6b87be1779b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:25.481235 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1249/2482] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:28:25.485969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1cd63-d8d1-4d12-9e43-8743edf3bbfa admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:25.497698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1cd63-d8d1-4d12-9e43-8743edf3bbfa admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:25.502874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77b8e7-e1e3-4952-9bfe-09d39806f914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:25.503486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77b8e7-e1e3-4952-9bfe-09d39806f914 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.503823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77b8e7-e1e3-4952-9bfe-09d39806f914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.504104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77b8e7-e1e3-4952-9bfe-09d39806f914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.504748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77b8e7-e1e3-4952-9bfe-09d39806f914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:25.506384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77b8e7-e1e3-4952-9bfe-09d39806f914 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:25.519841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1234/2483] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:28:25.523693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea04b7-640a-420e-b1d0-e6b87be1779b admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:25.534049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea04b7-640a-420e-b1d0-e6b87be1779b admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:25.538759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f53b95-501a-41d0-9207-a3fcd70ebe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:25.539429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53b95-501a-41d0-9207-a3fcd70ebe80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.539716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53b95-501a-41d0-9207-a3fcd70ebe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.540018 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f53b95-501a-41d0-9207-a3fcd70ebe80 admin admin] PATH_INFO: `/v3/projects/95515a2858124583ac138aa6350e4bc3/users/94c4cdfb0f264398a7d3b155a9523b11/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.555268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f53b95-501a-41d0-9207-a3fcd70ebe80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95515a2858124583ac138aa6350e4bc3, user_id=94c4cdfb0f264398a7d3b155a9523b11, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:25.557962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f53b95-501a-41d0-9207-a3fcd70ebe80 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:25.571953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1250/2484] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:28:25 2022] PUT /identity/v3/projects/95515a2858124583ac138aa6350e4bc3/users/94c4cdfb0f264398a7d3b155a9523b11/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:28:25.577319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29173117-5fb3-488f-8370-6add4c5997cc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.577691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29173117-5fb3-488f-8370-6add4c5997cc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.577967 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29173117-5fb3-488f-8370-6add4c5997cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.598979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-29173117-5fb3-488f-8370-6add4c5997cc None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:28:25.600301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-29173117-5fb3-488f-8370-6add4c5997cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:25.669546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1235/2485] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:28:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:28:25.679676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f53b95-501a-41d0-9207-a3fcd70ebe80 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:25.681488 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60f53b95-501a-41d0-9207-a3fcd70ebe80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:25.745695 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f53b95-501a-41d0-9207-a3fcd70ebe80 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:25.750695 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035ce2b-d376-41da-8cc6-ec49bd87bd82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:25.751305 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035ce2b-d376-41da-8cc6-ec49bd87bd82 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:25.751713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035ce2b-d376-41da-8cc6-ec49bd87bd82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:25.752029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035ce2b-d376-41da-8cc6-ec49bd87bd82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:25.765717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035ce2b-d376-41da-8cc6-ec49bd87bd82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:25.767186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3035ce2b-d376-41da-8cc6-ec49bd87bd82 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:25.777278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1251/2486] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:28:30.097308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8025b5c-2df6-4bc4-aae2-416780671112 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:30.097809 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8025b5c-2df6-4bc4-aae2-416780671112 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:30.097905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8025b5c-2df6-4bc4-aae2-416780671112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:30.135305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d8025b5c-2df6-4bc4-aae2-416780671112 None None] MFA Rules not processed for user `94c4cdfb0f264398a7d3b155a9523b11`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:28:30.136557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d8025b5c-2df6-4bc4-aae2-416780671112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:30.212728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1236/2487] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:28:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:28:30.217596 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035ce2b-d376-41da-8cc6-ec49bd87bd82 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:30.228013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035ce2b-d376-41da-8cc6-ec49bd87bd82 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:30.233410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84612b-18d6-4104-aa26-990fea45f68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:30.234017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84612b-18d6-4104-aa26-990fea45f68f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:30.234318 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84612b-18d6-4104-aa26-990fea45f68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:30.234587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84612b-18d6-4104-aa26-990fea45f68f admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:30.235229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84612b-18d6-4104-aa26-990fea45f68f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:30.237190 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84612b-18d6-4104-aa26-990fea45f68f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:30.267397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1252/2488] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:28:30 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:28:30.271629 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8025b5c-2df6-4bc4-aae2-416780671112 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:30.282717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8025b5c-2df6-4bc4-aae2-416780671112 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:30.344690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80916a88-16bb-4e92-b059-a89e5b85ae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:30.345479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80916a88-16bb-4e92-b059-a89e5b85ae3d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:30.345735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80916a88-16bb-4e92-b059-a89e5b85ae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:30.346086 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80916a88-16bb-4e92-b059-a89e5b85ae3d admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:30.346797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80916a88-16bb-4e92-b059-a89e5b85ae3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:30.348854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80916a88-16bb-4e92-b059-a89e5b85ae3d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:30.372590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1237/2489] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:28:30 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:28:30.376740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84612b-18d6-4104-aa26-990fea45f68f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:30.384016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:28:30.388286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84612b-18d6-4104-aa26-990fea45f68f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:30.394589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82010125-d3a9-4b8c-81c9-e1df56f7580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:30.395231 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82010125-d3a9-4b8c-81c9-e1df56f7580c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:30.395803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82010125-d3a9-4b8c-81c9-e1df56f7580c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:30.395939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82010125-d3a9-4b8c-81c9-e1df56f7580c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:30.396590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82010125-d3a9-4b8c-81c9-e1df56f7580c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:30.398625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82010125-d3a9-4b8c-81c9-e1df56f7580c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:30.411064 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1253/2490] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:28:30.415287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80916a88-16bb-4e92-b059-a89e5b85ae3d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:30.423586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:28:30.428401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80916a88-16bb-4e92-b059-a89e5b85ae3d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:30.435275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9dcd1-293d-435a-a13a-5adaf94f47a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:30.435978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9dcd1-293d-435a-a13a-5adaf94f47a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:30.436263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9dcd1-293d-435a-a13a-5adaf94f47a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:30.436544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9dcd1-293d-435a-a13a-5adaf94f47a5 admin admin] PATH_INFO: `/v3/projects/43c694babd814ea8bf1cdaf84f242a26/users/28bb5d5441e8430a97ff57f2d375d363/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:30.452848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9dcd1-293d-435a-a13a-5adaf94f47a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c694babd814ea8bf1cdaf84f242a26, user_id=28bb5d5441e8430a97ff57f2d375d363, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:30.455646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9dcd1-293d-435a-a13a-5adaf94f47a5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:30.471163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1238/2491] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:28:30 2022] PUT /identity/v3/projects/43c694babd814ea8bf1cdaf84f242a26/users/28bb5d5441e8430a97ff57f2d375d363/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:28:30.475321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82010125-d3a9-4b8c-81c9-e1df56f7580c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:30.486449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82010125-d3a9-4b8c-81c9-e1df56f7580c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:30.542495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae775627-317c-4674-b60e-10890492f4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:30.543107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae775627-317c-4674-b60e-10890492f4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:30.543382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae775627-317c-4674-b60e-10890492f4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:30.544214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae775627-317c-4674-b60e-10890492f4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:30.544856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae775627-317c-4674-b60e-10890492f4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:30.546487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae775627-317c-4674-b60e-10890492f4c4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:30.557448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1254/2492] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:28:30.561770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9dcd1-293d-435a-a13a-5adaf94f47a5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:30.572995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9dcd1-293d-435a-a13a-5adaf94f47a5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:30.578300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bde51-367a-440f-8721-d677e84603af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:30.578972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bde51-367a-440f-8721-d677e84603af admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:30.579251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bde51-367a-440f-8721-d677e84603af admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:30.579526 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bde51-367a-440f-8721-d677e84603af admin admin] PATH_INFO: `/v3/projects/43c694babd814ea8bf1cdaf84f242a26/users/28bb5d5441e8430a97ff57f2d375d363/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:30.581638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bde51-367a-440f-8721-d677e84603af admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c694babd814ea8bf1cdaf84f242a26, user_id=28bb5d5441e8430a97ff57f2d375d363, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:30.584415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bde51-367a-440f-8721-d677e84603af admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:30.595712 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1239/2493] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:28:30 2022] PUT /identity/v3/projects/43c694babd814ea8bf1cdaf84f242a26/users/28bb5d5441e8430a97ff57f2d375d363/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:28:35.081861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b682af-fdcf-4325-ba41-f12734faf4ee None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:35.082375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b682af-fdcf-4325-ba41-f12734faf4ee None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:35.082457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b682af-fdcf-4325-ba41-f12734faf4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:35.118924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-43b682af-fdcf-4325-ba41-f12734faf4ee None None] MFA Rules not processed for user `28bb5d5441e8430a97ff57f2d375d363`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:28:35.120489 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-43b682af-fdcf-4325-ba41-f12734faf4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:35.203776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1255/2494] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:28:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:28:35.224105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bde51-367a-440f-8721-d677e84603af admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:35.225871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-632bde51-367a-440f-8721-d677e84603af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:35.310376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bde51-367a-440f-8721-d677e84603af admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:35.320345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a8177-add3-446d-b412-f4543c39afa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:35.320984 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a8177-add3-446d-b412-f4543c39afa7 service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:35.321267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a8177-add3-446d-b412-f4543c39afa7 service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:35.321542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a8177-add3-446d-b412-f4543c39afa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:35.382621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a8177-add3-446d-b412-f4543c39afa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:35.384250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a8177-add3-446d-b412-f4543c39afa7 service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:35.393689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1240/2495] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:28:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:28:36.426139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157b540-4d42-4502-aaae-c181053ed2bc None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:36.426723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157b540-4d42-4502-aaae-c181053ed2bc None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:36.426723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157b540-4d42-4502-aaae-c181053ed2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:36.446722 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c157b540-4d42-4502-aaae-c181053ed2bc None None] MFA Rules not processed for user `94c4cdfb0f264398a7d3b155a9523b11`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:28:36.448275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c157b540-4d42-4502-aaae-c181053ed2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:36.456358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1256/2496] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:28:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:28:41.799349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a8177-add3-446d-b412-f4543c39afa7 service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:41.802252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e32a8177-add3-446d-b412-f4543c39afa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:41.878114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:28:41.898350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a8177-add3-446d-b412-f4543c39afa7 service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:41.902865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d97eba-0942-4f2f-8731-339e72d24d80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:41.903488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d97eba-0942-4f2f-8731-339e72d24d80 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:41.903807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d97eba-0942-4f2f-8731-339e72d24d80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:41.904091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d97eba-0942-4f2f-8731-339e72d24d80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:41.905963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e0d97eba-0942-4f2f-8731-339e72d24d80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:41.987601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d97eba-0942-4f2f-8731-339e72d24d80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:41.989071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d97eba-0942-4f2f-8731-339e72d24d80 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:41.998183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1241/2497] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:28:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:28:46.441672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=69 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:28:51.993945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=38 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:28:57.892775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157b540-4d42-4502-aaae-c181053ed2bc None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:28:57.894233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c157b540-4d42-4502-aaae-c181053ed2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:58.024141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157b540-4d42-4502-aaae-c181053ed2bc None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:28:58.030327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee784-c693-488a-bcf2-373598eb7a82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:28:58.030956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee784-c693-488a-bcf2-373598eb7a82 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:28:58.031244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee784-c693-488a-bcf2-373598eb7a82 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:28:58.031512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaee784-c693-488a-bcf2-373598eb7a82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:28:58.033322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ecaee784-c693-488a-bcf2-373598eb7a82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:28:58.048081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaee784-c693-488a-bcf2-373598eb7a82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:28:58.049851 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaee784-c693-488a-bcf2-373598eb7a82 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:28:58.063962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1257/2498] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:00.492634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d97eba-0942-4f2f-8731-339e72d24d80 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:00.557227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d97eba-0942-4f2f-8731-339e72d24d80 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:00.561451 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:00.562072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:00.562351 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:00.562621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:00.564386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:00.586854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:00.588312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:00.596262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1242/2499] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:04.413057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee784-c693-488a-bcf2-373598eb7a82 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:04.424246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaee784-c693-488a-bcf2-373598eb7a82 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:04.428839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d730a-604a-43c4-a270-affe895e96fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:04.430039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d730a-604a-43c4-a270-affe895e96fa service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:04.430326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d730a-604a-43c4-a270-affe895e96fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:04.430604 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d730a-604a-43c4-a270-affe895e96fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:04.432606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-230d730a-604a-43c4-a270-affe895e96fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:04.537439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d730a-604a-43c4-a270-affe895e96fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:04.539080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d730a-604a-43c4-a270-affe895e96fa service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:04.570313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1258/2500] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:07.890423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:07.892263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:07.911117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830b84c-0924-4bef-b11d-c649a8954aad service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:07.915885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:07.916519 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:07.916801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:07.917075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] PATH_INFO: `/v3/users/0fb9105e474a4a29904722b1e33d0155` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:07.918218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb9105e474a4a29904722b1e33d0155)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:07.920979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:07.970309 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb9105e474a4a29904722b1e33d0155'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:07.970991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb9105e474a4a29904722b1e33d0155'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:07.978518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb9105e474a4a29904722b1e33d0155'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:07.979204 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb9105e474a4a29904722b1e33d0155'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:07.987024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-956d2d8b-7977-4fad-ab6a-0df065f41f48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb9105e474a4a29904722b1e33d0155'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:08.008372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1243/2501] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:29:07 2022] DELETE /identity/v3/users/0fb9105e474a4a29904722b1e33d0155 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:08.199436 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d730a-604a-43c4-a270-affe895e96fa service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:08.205713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:08.219289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d730a-604a-43c4-a270-affe895e96fa service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:08.275685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:08.276339 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.276616 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.277595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] PATH_INFO: `/v3/projects/d3bd02ae79fd4edb9886b1b4ef3d761a` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.278010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3bd02ae79fd4edb9886b1b4ef3d761a)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:08.281169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:08.328395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3bd02ae79fd4edb9886b1b4ef3d761a', 'request_id': 'req-789fdd49-4605-4ca3-afa3-10140f8d2aa7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:08.335031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3bd02ae79fd4edb9886b1b4ef3d761a', 'request_id': 'req-789fdd49-4605-4ca3-afa3-10140f8d2aa7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:08.342519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3bd02ae79fd4edb9886b1b4ef3d761a', 'request_id': 'req-789fdd49-4605-4ca3-afa3-10140f8d2aa7'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:08.354129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] The token cache is being invalidate because project d3bd02ae79fd4edb9886b1b4ef3d761a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:08.354411 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:08.355034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-789fdd49-4605-4ca3-afa3-10140f8d2aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:08.356591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1259/2502] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:08 2022] DELETE /identity/v3/projects/d3bd02ae79fd4edb9886b1b4ef3d761a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:08.361569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30530dcf-3cd8-45d9-9597-a652da5b4454 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.361839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30530dcf-3cd8-45d9-9597-a652da5b4454 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.362179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30530dcf-3cd8-45d9-9597-a652da5b4454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.427170 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-30530dcf-3cd8-45d9-9597-a652da5b4454 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:08.428424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-30530dcf-3cd8-45d9-9597-a652da5b4454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:08.449377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:08.492593 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1244/2503] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:08.501328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a71e02-9f77-4cd1-8a8d-de67bed69037 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.501634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a71e02-9f77-4cd1-8a8d-de67bed69037 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.501936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a71e02-9f77-4cd1-8a8d-de67bed69037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.522156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d1a71e02-9f77-4cd1-8a8d-de67bed69037 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:08.523375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d1a71e02-9f77-4cd1-8a8d-de67bed69037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:08.531264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1260/2504] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:08.535601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30530dcf-3cd8-45d9-9597-a652da5b4454 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:08.545409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30530dcf-3cd8-45d9-9597-a652da5b4454 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:08.550022 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41882dc-c8d9-40e5-9d01-2f02523faee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:08.550623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41882dc-c8d9-40e5-9d01-2f02523faee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.550901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41882dc-c8d9-40e5-9d01-2f02523faee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.551172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41882dc-c8d9-40e5-9d01-2f02523faee0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.551903 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41882dc-c8d9-40e5-9d01-2f02523faee0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:29:08.552206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41882dc-c8d9-40e5-9d01-2f02523faee0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:08.553801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41882dc-c8d9-40e5-9d01-2f02523faee0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:08.565846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1245/2505] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:29:08.570306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a71e02-9f77-4cd1-8a8d-de67bed69037 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:08.580941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a71e02-9f77-4cd1-8a8d-de67bed69037 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:08.585593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d2569-23fc-49dc-8982-00f22b6b7ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:08.586189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d2569-23fc-49dc-8982-00f22b6b7ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.586470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d2569-23fc-49dc-8982-00f22b6b7ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.586742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d2569-23fc-49dc-8982-00f22b6b7ee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.587410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d2569-23fc-49dc-8982-00f22b6b7ee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:08.589479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d2569-23fc-49dc-8982-00f22b6b7ee9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:08.619298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1261/2506] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:08 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:08.620468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41882dc-c8d9-40e5-9d01-2f02523faee0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:08.633291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41882dc-c8d9-40e5-9d01-2f02523faee0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:08.691305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154fc0f-d394-4188-b4d7-085495bee87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:08.691945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154fc0f-d394-4188-b4d7-085495bee87c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.692317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154fc0f-d394-4188-b4d7-085495bee87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.692488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154fc0f-d394-4188-b4d7-085495bee87c admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.693112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154fc0f-d394-4188-b4d7-085495bee87c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:08.695036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154fc0f-d394-4188-b4d7-085495bee87c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:08.720258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1246/2507] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:29:08 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:29:08.724476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d2569-23fc-49dc-8982-00f22b6b7ee9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:08.734892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d2569-23fc-49dc-8982-00f22b6b7ee9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:08.739925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd8c55-c0fe-43d7-84f3-4cc177736310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:08.740527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd8c55-c0fe-43d7-84f3-4cc177736310 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.740810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd8c55-c0fe-43d7-84f3-4cc177736310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.741086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd8c55-c0fe-43d7-84f3-4cc177736310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.741861 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd8c55-c0fe-43d7-84f3-4cc177736310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:08.743772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd8c55-c0fe-43d7-84f3-4cc177736310 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:08.754731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1262/2508] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:08.758771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154fc0f-d394-4188-b4d7-085495bee87c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:08.769674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154fc0f-d394-4188-b4d7-085495bee87c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:08.774347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764168d-0344-403d-99c9-ff3570ec12ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:08.774933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764168d-0344-403d-99c9-ff3570ec12ee admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.775217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764168d-0344-403d-99c9-ff3570ec12ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.775494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764168d-0344-403d-99c9-ff3570ec12ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.776167 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764168d-0344-403d-99c9-ff3570ec12ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:08.777776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764168d-0344-403d-99c9-ff3570ec12ee admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:08.788518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1247/2509] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:08.792846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd8c55-c0fe-43d7-84f3-4cc177736310 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:08.802973 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd8c55-c0fe-43d7-84f3-4cc177736310 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:08.807588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e15c-27b6-4e04-986e-18aadb12f615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:08.808359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e15c-27b6-4e04-986e-18aadb12f615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.808719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e15c-27b6-4e04-986e-18aadb12f615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.809007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e15c-27b6-4e04-986e-18aadb12f615 admin admin] PATH_INFO: `/v3/projects/787f3db976f241908f0d9451e962cfc5/users/230c458b64a247018d8385ab3e5cd574/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.826367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1e15c-27b6-4e04-986e-18aadb12f615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=787f3db976f241908f0d9451e962cfc5, user_id=230c458b64a247018d8385ab3e5cd574, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:08.829111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1e15c-27b6-4e04-986e-18aadb12f615 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:08.844219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1263/2510] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:08 2022] PUT /identity/v3/projects/787f3db976f241908f0d9451e962cfc5/users/230c458b64a247018d8385ab3e5cd574/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:08.854470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab12bc-6d3f-4429-a43a-ed268bf70359 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:08.854754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab12bc-6d3f-4429-a43a-ed268bf70359 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:08.855025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab12bc-6d3f-4429-a43a-ed268bf70359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:08.883233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-85ab12bc-6d3f-4429-a43a-ed268bf70359 None None] MFA Rules not processed for user `230c458b64a247018d8385ab3e5cd574`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:08.884380 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-85ab12bc-6d3f-4429-a43a-ed268bf70359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:08.948056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1248/2511] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:08.952921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e15c-27b6-4e04-986e-18aadb12f615 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:08.985002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:09.017416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e15c-27b6-4e04-986e-18aadb12f615 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:09.022092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58caa87-1e11-4b1a-be49-330bd17611c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:09.022695 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58caa87-1e11-4b1a-be49-330bd17611c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:09.022977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58caa87-1e11-4b1a-be49-330bd17611c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:09.023250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58caa87-1e11-4b1a-be49-330bd17611c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:09.023961 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58caa87-1e11-4b1a-be49-330bd17611c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:09.025872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58caa87-1e11-4b1a-be49-330bd17611c6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:09.053778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1264/2512] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:08 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:09.056428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab12bc-6d3f-4429-a43a-ed268bf70359 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:09.067801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab12bc-6d3f-4429-a43a-ed268bf70359 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:09.085278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:09.126614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc2438-2db1-4583-9169-d9e8074e5d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:09.127215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc2438-2db1-4583-9169-d9e8074e5d41 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:09.127489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc2438-2db1-4583-9169-d9e8074e5d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:09.127784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc2438-2db1-4583-9169-d9e8074e5d41 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:09.128409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc2438-2db1-4583-9169-d9e8074e5d41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:09.130304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc2438-2db1-4583-9169-d9e8074e5d41 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:09.150940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1249/2513] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:29:09 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:09.155076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58caa87-1e11-4b1a-be49-330bd17611c6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:09.166054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58caa87-1e11-4b1a-be49-330bd17611c6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:09.171455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccf1e8-b730-46b6-b663-41fe7ccb19cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:09.172170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf1e8-b730-46b6-b663-41fe7ccb19cb admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:09.172553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf1e8-b730-46b6-b663-41fe7ccb19cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:09.172891 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf1e8-b730-46b6-b663-41fe7ccb19cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:09.173600 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccf1e8-b730-46b6-b663-41fe7ccb19cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:09.175266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccf1e8-b730-46b6-b663-41fe7ccb19cb admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:09.187213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1265/2514] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:09.191724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc2438-2db1-4583-9169-d9e8074e5d41 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:09.202178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc2438-2db1-4583-9169-d9e8074e5d41 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:09.207194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c5a76-c670-4cb8-9156-dd4ac41340fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:09.207907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c5a76-c670-4cb8-9156-dd4ac41340fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:09.208189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c5a76-c670-4cb8-9156-dd4ac41340fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:09.208473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c5a76-c670-4cb8-9156-dd4ac41340fa admin admin] PATH_INFO: `/v3/projects/2d35f0db710e47d58083b1a3f031dae8/users/6f92fa404f804f188560b2d0d1c6b7bc/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:09.224021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c5a76-c670-4cb8-9156-dd4ac41340fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d35f0db710e47d58083b1a3f031dae8, user_id=6f92fa404f804f188560b2d0d1c6b7bc, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:09.227774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c5a76-c670-4cb8-9156-dd4ac41340fa admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:09.244180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1250/2515] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:09 2022] PUT /identity/v3/projects/2d35f0db710e47d58083b1a3f031dae8/users/6f92fa404f804f188560b2d0d1c6b7bc/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:09.249097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccf1e8-b730-46b6-b663-41fe7ccb19cb admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:09.261030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccf1e8-b730-46b6-b663-41fe7ccb19cb admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:09.326404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d976ce8-7d62-4776-877c-851d87fdc29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:09.328031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976ce8-7d62-4776-877c-851d87fdc29b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:09.328342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976ce8-7d62-4776-877c-851d87fdc29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:09.328625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d976ce8-7d62-4776-877c-851d87fdc29b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:09.329307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d976ce8-7d62-4776-877c-851d87fdc29b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:09.330929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d976ce8-7d62-4776-877c-851d87fdc29b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:09.347087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1266/2516] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:09.352253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c5a76-c670-4cb8-9156-dd4ac41340fa admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:09.366582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c5a76-c670-4cb8-9156-dd4ac41340fa admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:09.372136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22708c-4422-4b12-bdfa-e06136e07f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:09.372867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22708c-4422-4b12-bdfa-e06136e07f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:09.373148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22708c-4422-4b12-bdfa-e06136e07f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:09.373421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c22708c-4422-4b12-bdfa-e06136e07f7a admin admin] PATH_INFO: `/v3/projects/2d35f0db710e47d58083b1a3f031dae8/users/6f92fa404f804f188560b2d0d1c6b7bc/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:09.375536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22708c-4422-4b12-bdfa-e06136e07f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d35f0db710e47d58083b1a3f031dae8, user_id=6f92fa404f804f188560b2d0d1c6b7bc, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:09.378277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c22708c-4422-4b12-bdfa-e06136e07f7a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:09.390388 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1251/2517] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:09 2022] PUT /identity/v3/projects/2d35f0db710e47d58083b1a3f031dae8/users/6f92fa404f804f188560b2d0d1c6b7bc/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:09.407929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58c690-6b7b-4dcf-9751-b3b02291320f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:09.407929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58c690-6b7b-4dcf-9751-b3b02291320f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:09.407929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58c690-6b7b-4dcf-9751-b3b02291320f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:09.437468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4e58c690-6b7b-4dcf-9751-b3b02291320f None None] MFA Rules not processed for user `6f92fa404f804f188560b2d0d1c6b7bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:09.438703 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e58c690-6b7b-4dcf-9751-b3b02291320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:09.512403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1267/2518] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:09.524610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22708c-4422-4b12-bdfa-e06136e07f7a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:09.526304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3c22708c-4422-4b12-bdfa-e06136e07f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:09.608879 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c22708c-4422-4b12-bdfa-e06136e07f7a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:09.612588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c55b2-e4be-4c36-b909-354d2ee4737a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:09.613188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c55b2-e4be-4c36-b909-354d2ee4737a service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:09.613473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c55b2-e4be-4c36-b909-354d2ee4737a service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:09.613743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c55b2-e4be-4c36-b909-354d2ee4737a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:09.622681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c55b2-e4be-4c36-b909-354d2ee4737a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:09.624180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c55b2-e4be-4c36-b909-354d2ee4737a service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:09.633537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1252/2519] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:09.955213 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58c690-6b7b-4dcf-9751-b3b02291320f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:09.964459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58c690-6b7b-4dcf-9751-b3b02291320f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:09.969702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18abd0-aea1-4820-bb46-c468877660c4 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f92fa404f804f188560b2d0d1c6b7bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d35f0db710e47d58083b1a3f031dae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:09.970304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18abd0-aea1-4820-bb46-c468877660c4 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:09.970597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18abd0-aea1-4820-bb46-c468877660c4 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:09.970876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18abd0-aea1-4820-bb46-c468877660c4 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:09.971526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18abd0-aea1-4820-bb46-c468877660c4 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:29:09.971868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18abd0-aea1-4820-bb46-c468877660c4 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:09.973486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18abd0-aea1-4820-bb46-c468877660c4 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:09.985015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1268/2520] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:29:09.989299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c55b2-e4be-4c36-b909-354d2ee4737a service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:09.998863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c55b2-e4be-4c36-b909-354d2ee4737a service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:10.003475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870abbe-2849-4f5b-af92-299a8244e5e3 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f92fa404f804f188560b2d0d1c6b7bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d35f0db710e47d58083b1a3f031dae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:10.004105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870abbe-2849-4f5b-af92-299a8244e5e3 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:10.004392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870abbe-2849-4f5b-af92-299a8244e5e3 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:10.004668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870abbe-2849-4f5b-af92-299a8244e5e3 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:10.005295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870abbe-2849-4f5b-af92-299a8244e5e3 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:10.007199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870abbe-2849-4f5b-af92-299a8244e5e3 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:10.037899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1253/2521] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:09 2022] POST /identity/v3/projects => generated 353 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:10.041972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18abd0-aea1-4820-bb46-c468877660c4 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:10.053619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18abd0-aea1-4820-bb46-c468877660c4 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:10.061919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:10.112144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f92fa404f804f188560b2d0d1c6b7bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d35f0db710e47d58083b1a3f031dae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:10.112753 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:10.113040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:10.113316 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:10.113959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:29:10.114265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:10.115893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:10.128251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1269/2522] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:29:10.166793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870abbe-2849-4f5b-af92-299a8244e5e3 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:10.177155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870abbe-2849-4f5b-af92-299a8244e5e3 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:10.181637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f92fa404f804f188560b2d0d1c6b7bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2d35f0db710e47d58083b1a3f031dae8', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:10.182254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:10.182539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:10.182817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] PATH_INFO: `/v3/projects/d6e89591dce64951a805d65548ddcf9d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:10.183963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d6e89591dce64951a805d65548ddcf9d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:10.186384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:10.218976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:10.235491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e89591dce64951a805d65548ddcf9d', 'request_id': 'req-d771cb40-7b27-4a81-a834-168dcabfc4eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.242263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e89591dce64951a805d65548ddcf9d', 'request_id': 'req-d771cb40-7b27-4a81-a834-168dcabfc4eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.249613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e89591dce64951a805d65548ddcf9d', 'request_id': 'req-d771cb40-7b27-4a81-a834-168dcabfc4eb'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.260020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] The token cache is being invalidate because project d6e89591dce64951a805d65548ddcf9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:10.260308 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.260955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d771cb40-7b27-4a81-a834-168dcabfc4eb tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.262464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1254/2523] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:10 2022] DELETE /identity/v3/projects/d6e89591dce64951a805d65548ddcf9d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:10.454897 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:10.456673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:10.522856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dd5f8-b73e-477f-bc1f-6a013ff79377 tempest-VolumeQuotaClassesTest-1294683100 tempest-VolumeQuotaClassesTest-1294683100-project-admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:10.579225 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:10.579940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:10.580173 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:10.580453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] PATH_INFO: `/v3/users/230c458b64a247018d8385ab3e5cd574` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:10.581689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=230c458b64a247018d8385ab3e5cd574)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:10.583582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:10.637355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230c458b64a247018d8385ab3e5cd574'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.645581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230c458b64a247018d8385ab3e5cd574'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.646320 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230c458b64a247018d8385ab3e5cd574'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.646938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230c458b64a247018d8385ab3e5cd574'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.648302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:10.656984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230c458b64a247018d8385ab3e5cd574'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:10.675776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1270/2524] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:29:10 2022] DELETE /identity/v3/users/230c458b64a247018d8385ab3e5cd574 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:10.680197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0cd2f-53b6-4e8e-b68c-995593fca32f None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:10.680484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0cd2f-53b6-4e8e-b68c-995593fca32f None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:10.680800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0cd2f-53b6-4e8e-b68c-995593fca32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:10.700877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-46e0cd2f-53b6-4e8e-b68c-995593fca32f None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:10.702082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-46e0cd2f-53b6-4e8e-b68c-995593fca32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:10.775522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1255/2525] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:10.785867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:10.787333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:10.844989 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1894bd0-3cc0-445e-8d06-43c52602a988 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:10.848686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73918c53-6fb8-4bc5-907a-1eca6a390d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:10.849301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73918c53-6fb8-4bc5-907a-1eca6a390d4a service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:10.849579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73918c53-6fb8-4bc5-907a-1eca6a390d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:10.849853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73918c53-6fb8-4bc5-907a-1eca6a390d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:10.859719 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73918c53-6fb8-4bc5-907a-1eca6a390d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:10.861189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73918c53-6fb8-4bc5-907a-1eca6a390d4a service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:10.871835 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1271/2526] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:11.159268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0cd2f-53b6-4e8e-b68c-995593fca32f None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:11.160768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-46e0cd2f-53b6-4e8e-b68c-995593fca32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:11.172812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0cd2f-53b6-4e8e-b68c-995593fca32f None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:11.177757 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:11.178428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:11.178845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:11.179122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] PATH_INFO: `/v3/projects/787f3db976f241908f0d9451e962cfc5` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:11.180284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=787f3db976f241908f0d9451e962cfc5)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:11.182932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:11.230771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '787f3db976f241908f0d9451e962cfc5', 'request_id': 'req-a2880c52-6380-4086-ad4f-b21ab4f1d069'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.237442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '787f3db976f241908f0d9451e962cfc5', 'request_id': 'req-a2880c52-6380-4086-ad4f-b21ab4f1d069'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.245923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '787f3db976f241908f0d9451e962cfc5', 'request_id': 'req-a2880c52-6380-4086-ad4f-b21ab4f1d069'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.260558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] The token cache is being invalidate because project 787f3db976f241908f0d9451e962cfc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:11.260898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.261805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.264256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1256/2527] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:11 2022] DELETE /identity/v3/projects/787f3db976f241908f0d9451e962cfc5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:11.268733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73918c53-6fb8-4bc5-907a-1eca6a390d4a service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:11.280117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73918c53-6fb8-4bc5-907a-1eca6a390d4a service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:11.337282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:11.338266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:11.338266 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:11.338547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] PATH_INFO: `/v3/users/6f92fa404f804f188560b2d0d1c6b7bc` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:11.339782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f92fa404f804f188560b2d0d1c6b7bc)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:11.342570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:11.390412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f92fa404f804f188560b2d0d1c6b7bc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.398505 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f92fa404f804f188560b2d0d1c6b7bc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.399286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f92fa404f804f188560b2d0d1c6b7bc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.399964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f92fa404f804f188560b2d0d1c6b7bc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.407661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-27a185c9-3f55-488e-b2b1-cf48d6b5f5b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f92fa404f804f188560b2d0d1c6b7bc'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.417784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:11.427721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1272/2528] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:29:11 2022] DELETE /identity/v3/users/6f92fa404f804f188560b2d0d1c6b7bc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:11.717042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:11.718592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:11.739662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2880c52-6380-4086-ad4f-b21ab4f1d069 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:11.786686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:11.807539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:11.808161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:11.808471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:11.808748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] PATH_INFO: `/v3/projects/2d35f0db710e47d58083b1a3f031dae8` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:11.810623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d35f0db710e47d58083b1a3f031dae8)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:11.812860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:11.867354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d35f0db710e47d58083b1a3f031dae8', 'request_id': 'req-103326cf-7155-489f-9d9e-246ab3a7984d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.874945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d35f0db710e47d58083b1a3f031dae8', 'request_id': 'req-103326cf-7155-489f-9d9e-246ab3a7984d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.884424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d35f0db710e47d58083b1a3f031dae8', 'request_id': 'req-103326cf-7155-489f-9d9e-246ab3a7984d'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.899270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] The token cache is being invalidate because project 2d35f0db710e47d58083b1a3f031dae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:11.899566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.901164 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-103326cf-7155-489f-9d9e-246ab3a7984d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:11.902908 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1257/2529] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:11 2022] DELETE /identity/v3/projects/2d35f0db710e47d58083b1a3f031dae8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:11.907770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d24da-ceb9-4e3d-8468-21f8115d8349 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:11.908603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d24da-ceb9-4e3d-8468-21f8115d8349 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:11.908875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d24da-ceb9-4e3d-8468-21f8115d8349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:11.929679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-be7d24da-ceb9-4e3d-8468-21f8115d8349 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:11.930900 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-be7d24da-ceb9-4e3d-8468-21f8115d8349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:11.992843 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1273/2530] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:12.001377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a297bef-324d-46b8-8464-701b66ae183b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.001666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a297bef-324d-46b8-8464-701b66ae183b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.001945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a297bef-324d-46b8-8464-701b66ae183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.022291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7a297bef-324d-46b8-8464-701b66ae183b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:12.023415 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7a297bef-324d-46b8-8464-701b66ae183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:12.031230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1258/2531] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:12.035389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d24da-ceb9-4e3d-8468-21f8115d8349 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.045353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d24da-ceb9-4e3d-8468-21f8115d8349 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.049893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb60917-56e0-44b4-9a86-45f6b761cf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.050480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb60917-56e0-44b4-9a86-45f6b761cf55 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.050756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb60917-56e0-44b4-9a86-45f6b761cf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.051027 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb60917-56e0-44b4-9a86-45f6b761cf55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.051663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb60917-56e0-44b4-9a86-45f6b761cf55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:29:12.051991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb60917-56e0-44b4-9a86-45f6b761cf55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.053571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb60917-56e0-44b4-9a86-45f6b761cf55 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.065919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1274/2532] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:29:12.070243 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a297bef-324d-46b8-8464-701b66ae183b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.079669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a297bef-324d-46b8-8464-701b66ae183b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.084314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3017b90-d803-48c8-8387-a3624768ed01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.084915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3017b90-d803-48c8-8387-a3624768ed01 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.085198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3017b90-d803-48c8-8387-a3624768ed01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.085470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3017b90-d803-48c8-8387-a3624768ed01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.086102 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3017b90-d803-48c8-8387-a3624768ed01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.088012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3017b90-d803-48c8-8387-a3624768ed01 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.114900 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1259/2533] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:12 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:12.119319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb60917-56e0-44b4-9a86-45f6b761cf55 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.129908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb60917-56e0-44b4-9a86-45f6b761cf55 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.188603 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac43af2-439c-4d47-a588-3076dfefb8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.189562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac43af2-439c-4d47-a588-3076dfefb8fe admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.190088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac43af2-439c-4d47-a588-3076dfefb8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.190362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac43af2-439c-4d47-a588-3076dfefb8fe admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.191006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac43af2-439c-4d47-a588-3076dfefb8fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.192956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac43af2-439c-4d47-a588-3076dfefb8fe admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.214620 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1275/2534] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:29:12 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:12.218733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3017b90-d803-48c8-8387-a3624768ed01 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.228930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3017b90-d803-48c8-8387-a3624768ed01 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.233998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcdfe6-4f58-46a8-989b-691c2aa13fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.234584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcdfe6-4f58-46a8-989b-691c2aa13fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.234859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcdfe6-4f58-46a8-989b-691c2aa13fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.235126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcdfe6-4f58-46a8-989b-691c2aa13fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.235796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcdfe6-4f58-46a8-989b-691c2aa13fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.237351 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcdfe6-4f58-46a8-989b-691c2aa13fb3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.247713 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1260/2535] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:12.252721 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac43af2-439c-4d47-a588-3076dfefb8fe admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.263037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac43af2-439c-4d47-a588-3076dfefb8fe admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.267707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e021952-1129-4b20-a5a4-1ae2cc10c830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.268330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e021952-1129-4b20-a5a4-1ae2cc10c830 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.268609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e021952-1129-4b20-a5a4-1ae2cc10c830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.268882 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e021952-1129-4b20-a5a4-1ae2cc10c830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.269525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e021952-1129-4b20-a5a4-1ae2cc10c830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.271247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e021952-1129-4b20-a5a4-1ae2cc10c830 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.282453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1276/2536] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:29:12.286567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcdfe6-4f58-46a8-989b-691c2aa13fb3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.297135 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcdfe6-4f58-46a8-989b-691c2aa13fb3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.301749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b37e14-397b-4825-a551-6818ac5cf2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.302393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b37e14-397b-4825-a551-6818ac5cf2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.302670 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b37e14-397b-4825-a551-6818ac5cf2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.302940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b37e14-397b-4825-a551-6818ac5cf2f5 admin admin] PATH_INFO: `/v3/projects/3af9242d8d644ce6a462010277a72a11/users/aaec042456ed4823bec8ae9483f9a02f/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.318032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b37e14-397b-4825-a551-6818ac5cf2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af9242d8d644ce6a462010277a72a11, user_id=aaec042456ed4823bec8ae9483f9a02f, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.320751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b37e14-397b-4825-a551-6818ac5cf2f5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.334025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1261/2537] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:12 2022] PUT /identity/v3/projects/3af9242d8d644ce6a462010277a72a11/users/aaec042456ed4823bec8ae9483f9a02f/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:12.347201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c76edc-b732-45e6-bb88-29a2e2f30962 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.347490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c76edc-b732-45e6-bb88-29a2e2f30962 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.347786 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c76edc-b732-45e6-bb88-29a2e2f30962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.375442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-60c76edc-b732-45e6-bb88-29a2e2f30962 None None] MFA Rules not processed for user `aaec042456ed4823bec8ae9483f9a02f`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:12.376696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-60c76edc-b732-45e6-bb88-29a2e2f30962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:12.423495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:12.443563 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1277/2538] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:29:12.449116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b37e14-397b-4825-a551-6818ac5cf2f5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.511354 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b37e14-397b-4825-a551-6818ac5cf2f5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.516067 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f7881-6cf6-4eae-b2a7-cddf64f62c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.516673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f7881-6cf6-4eae-b2a7-cddf64f62c96 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.516955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f7881-6cf6-4eae-b2a7-cddf64f62c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.517236 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f7881-6cf6-4eae-b2a7-cddf64f62c96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.517874 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f7881-6cf6-4eae-b2a7-cddf64f62c96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.519790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f7881-6cf6-4eae-b2a7-cddf64f62c96 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.531442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:12.550635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1262/2539] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:12 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:12.554551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c76edc-b732-45e6-bb88-29a2e2f30962 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.565155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c76edc-b732-45e6-bb88-29a2e2f30962 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.621460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382b7e4-cc92-4016-9fa5-e18337d96752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.622068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382b7e4-cc92-4016-9fa5-e18337d96752 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.622353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382b7e4-cc92-4016-9fa5-e18337d96752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.622629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382b7e4-cc92-4016-9fa5-e18337d96752 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.623275 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382b7e4-cc92-4016-9fa5-e18337d96752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.626470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382b7e4-cc92-4016-9fa5-e18337d96752 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.654373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1278/2540] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:29:12 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:12.659084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f7881-6cf6-4eae-b2a7-cddf64f62c96 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.669592 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f7881-6cf6-4eae-b2a7-cddf64f62c96 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.674181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534aa4a9-04f6-48cb-8ae4-09096cb657cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.674782 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534aa4a9-04f6-48cb-8ae4-09096cb657cf admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.675060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534aa4a9-04f6-48cb-8ae4-09096cb657cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.675331 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534aa4a9-04f6-48cb-8ae4-09096cb657cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.676007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534aa4a9-04f6-48cb-8ae4-09096cb657cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.677599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534aa4a9-04f6-48cb-8ae4-09096cb657cf admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.687719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1263/2541] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:12.691873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382b7e4-cc92-4016-9fa5-e18337d96752 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.702832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382b7e4-cc92-4016-9fa5-e18337d96752 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.707545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04208-7e6f-4b4f-8f68-2ea97f8b620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.708212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04208-7e6f-4b4f-8f68-2ea97f8b620a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.708495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04208-7e6f-4b4f-8f68-2ea97f8b620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.708767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04208-7e6f-4b4f-8f68-2ea97f8b620a admin admin] PATH_INFO: `/v3/projects/7636aed9ebaf4b2e8148ed7d77b95eac/users/d873c53a116047eab3274cf2371dcef8/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.724282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd04208-7e6f-4b4f-8f68-2ea97f8b620a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7636aed9ebaf4b2e8148ed7d77b95eac, user_id=d873c53a116047eab3274cf2371dcef8, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.726958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd04208-7e6f-4b4f-8f68-2ea97f8b620a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.739956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1279/2542] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:12 2022] PUT /identity/v3/projects/7636aed9ebaf4b2e8148ed7d77b95eac/users/d873c53a116047eab3274cf2371dcef8/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:12.744256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534aa4a9-04f6-48cb-8ae4-09096cb657cf admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.757896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534aa4a9-04f6-48cb-8ae4-09096cb657cf admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.827148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3122dc-7fd0-4d56-b16c-ace646275beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.827796 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3122dc-7fd0-4d56-b16c-ace646275beb admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.828083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3122dc-7fd0-4d56-b16c-ace646275beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.828542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3122dc-7fd0-4d56-b16c-ace646275beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.829163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3122dc-7fd0-4d56-b16c-ace646275beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.830799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3122dc-7fd0-4d56-b16c-ace646275beb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.842048 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1264/2543] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:12.846999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04208-7e6f-4b4f-8f68-2ea97f8b620a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:12.860729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04208-7e6f-4b4f-8f68-2ea97f8b620a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:12.865808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c8a0-425a-417a-89e6-09176ea92ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:12.866484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c8a0-425a-417a-89e6-09176ea92ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.866763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c8a0-425a-417a-89e6-09176ea92ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.867035 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2c8a0-425a-417a-89e6-09176ea92ae1 admin admin] PATH_INFO: `/v3/projects/7636aed9ebaf4b2e8148ed7d77b95eac/users/d873c53a116047eab3274cf2371dcef8/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.869023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2c8a0-425a-417a-89e6-09176ea92ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7636aed9ebaf4b2e8148ed7d77b95eac, user_id=d873c53a116047eab3274cf2371dcef8, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:12.871702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2c8a0-425a-417a-89e6-09176ea92ae1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:12.883250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1280/2544] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:12 2022] PUT /identity/v3/projects/7636aed9ebaf4b2e8148ed7d77b95eac/users/d873c53a116047eab3274cf2371dcef8/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:12.895315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d005d4-d6dd-4d88-bf9c-fffc81eee07c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:12.895708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d005d4-d6dd-4d88-bf9c-fffc81eee07c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:12.896006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d005d4-d6dd-4d88-bf9c-fffc81eee07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:12.925824 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a4d005d4-d6dd-4d88-bf9c-fffc81eee07c None None] MFA Rules not processed for user `d873c53a116047eab3274cf2371dcef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:12.927039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a4d005d4-d6dd-4d88-bf9c-fffc81eee07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:13.006186 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1265/2545] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:13.018094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c8a0-425a-417a-89e6-09176ea92ae1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:13.019933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8dc2c8a0-425a-417a-89e6-09176ea92ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:13.084633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2c8a0-425a-417a-89e6-09176ea92ae1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:13.088800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32677e6e-3ece-4cbd-8131-2f654f81a637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:13.089565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32677e6e-3ece-4cbd-8131-2f654f81a637 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:13.089764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32677e6e-3ece-4cbd-8131-2f654f81a637 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:13.090034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32677e6e-3ece-4cbd-8131-2f654f81a637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:13.100272 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32677e6e-3ece-4cbd-8131-2f654f81a637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:13.101736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32677e6e-3ece-4cbd-8131-2f654f81a637 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:13.112116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1281/2546] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:13.225770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d005d4-d6dd-4d88-bf9c-fffc81eee07c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:13.235471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d005d4-d6dd-4d88-bf9c-fffc81eee07c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:13.239136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a43655-de20-450e-a063-9292d1c99266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:13.239739 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a43655-de20-450e-a063-9292d1c99266 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:13.240053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a43655-de20-450e-a063-9292d1c99266 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:13.240327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a43655-de20-450e-a063-9292d1c99266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:13.297827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a43655-de20-450e-a063-9292d1c99266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:13.299259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a43655-de20-450e-a063-9292d1c99266 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:13.303912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:13.309110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1266/2547] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:15.924643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32677e6e-3ece-4cbd-8131-2f654f81a637 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:15.969816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:15.987784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32677e6e-3ece-4cbd-8131-2f654f81a637 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:15.992321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:15.992943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:15.993222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:15.993495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] PATH_INFO: `/v3/users/aaec042456ed4823bec8ae9483f9a02f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:15.994612 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaec042456ed4823bec8ae9483f9a02f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:15.996611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:16.042606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaec042456ed4823bec8ae9483f9a02f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.050261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaec042456ed4823bec8ae9483f9a02f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.050874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaec042456ed4823bec8ae9483f9a02f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.051478 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaec042456ed4823bec8ae9483f9a02f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.058294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaec042456ed4823bec8ae9483f9a02f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.076538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1282/2548] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:29:15 2022] DELETE /identity/v3/users/aaec042456ed4823bec8ae9483f9a02f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:16.080976 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff3a25-f5dc-45b1-8177-56cf6dad2487 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:16.081470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff3a25-f5dc-45b1-8177-56cf6dad2487 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:16.081575 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff3a25-f5dc-45b1-8177-56cf6dad2487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:16.100539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-acff3a25-f5dc-45b1-8177-56cf6dad2487 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:16.101950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-acff3a25-f5dc-45b1-8177-56cf6dad2487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:16.159389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1267/2549] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:16.169242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:16.170654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:16.228072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a64eb3-bd87-4806-8f05-e2bb7c7dcc8d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:16.231778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1d1d7-cf9d-46fa-bcf4-5b49f76deb30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:16.232399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1d1d7-cf9d-46fa-bcf4-5b49f76deb30 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:16.232683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1d1d7-cf9d-46fa-bcf4-5b49f76deb30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:16.232956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1d1d7-cf9d-46fa-bcf4-5b49f76deb30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:16.241976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1d1d7-cf9d-46fa-bcf4-5b49f76deb30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:16.243408 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1d1d7-cf9d-46fa-bcf4-5b49f76deb30 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:16.253323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1283/2550] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:16.558406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff3a25-f5dc-45b1-8177-56cf6dad2487 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:16.560441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-acff3a25-f5dc-45b1-8177-56cf6dad2487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:16.579079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff3a25-f5dc-45b1-8177-56cf6dad2487 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:16.585313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:16.586291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:16.586815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:16.587159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] PATH_INFO: `/v3/projects/3af9242d8d644ce6a462010277a72a11` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:16.588626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3af9242d8d644ce6a462010277a72a11)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:16.590684 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:16.645951 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3af9242d8d644ce6a462010277a72a11', 'request_id': 'req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.652737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3af9242d8d644ce6a462010277a72a11', 'request_id': 'req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.662524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3af9242d8d644ce6a462010277a72a11', 'request_id': 'req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.677987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] The token cache is being invalidate because project 3af9242d8d644ce6a462010277a72a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:16.678342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.678946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.681223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1268/2551] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:16 2022] DELETE /identity/v3/projects/3af9242d8d644ce6a462010277a72a11 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:29:16.685818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1d1d7-cf9d-46fa-bcf4-5b49f76deb30 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:16.697377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1d1d7-cf9d-46fa-bcf4-5b49f76deb30 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:16.763312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:16.764022 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:16.764702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:16.764702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] PATH_INFO: `/v3/users/d873c53a116047eab3274cf2371dcef8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:16.765733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d873c53a116047eab3274cf2371dcef8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:16.767665 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:16.775464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:16.820151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd873c53a116047eab3274cf2371dcef8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.829381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd873c53a116047eab3274cf2371dcef8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.830028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd873c53a116047eab3274cf2371dcef8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.830642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd873c53a116047eab3274cf2371dcef8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.841807 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c445f7c1-0874-4853-9209-4091aed58d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd873c53a116047eab3274cf2371dcef8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:16.861552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1284/2552] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:29:16 2022] DELETE /identity/v3/users/d873c53a116047eab3274cf2371dcef8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:17.139076 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:17.140539 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:17.155833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94657bbe-4fb5-4f5b-b653-e28d9cdb79ca admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:17.214292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:17.214918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.215203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.215479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] PATH_INFO: `/v3/projects/7636aed9ebaf4b2e8148ed7d77b95eac` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.217237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7636aed9ebaf4b2e8148ed7d77b95eac)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:17.219145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:17.249362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:17.267214 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7636aed9ebaf4b2e8148ed7d77b95eac', 'request_id': 'req-922366aa-0bf8-4852-b5af-2da758916387'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:17.278117 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7636aed9ebaf4b2e8148ed7d77b95eac', 'request_id': 'req-922366aa-0bf8-4852-b5af-2da758916387'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:17.287510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7636aed9ebaf4b2e8148ed7d77b95eac', 'request_id': 'req-922366aa-0bf8-4852-b5af-2da758916387'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:17.299491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] The token cache is being invalidate because project 7636aed9ebaf4b2e8148ed7d77b95eac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:17.299917 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:17.300722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-922366aa-0bf8-4852-b5af-2da758916387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:17.302401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1269/2553] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:17 2022] DELETE /identity/v3/projects/7636aed9ebaf4b2e8148ed7d77b95eac => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:17.307484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefc1fa-ab80-41f6-9233-e4ff828272bb None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.308046 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefc1fa-ab80-41f6-9233-e4ff828272bb None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.308145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefc1fa-ab80-41f6-9233-e4ff828272bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.328830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1fefc1fa-ab80-41f6-9233-e4ff828272bb None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:17.330044 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1fefc1fa-ab80-41f6-9233-e4ff828272bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:17.389569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1285/2554] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:17.397514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bbc32-9a12-4445-9768-1332939bf333 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.397805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bbc32-9a12-4445-9768-1332939bf333 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.398085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bbc32-9a12-4445-9768-1332939bf333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.418396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-222bbc32-9a12-4445-9768-1332939bf333 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:17.419620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-222bbc32-9a12-4445-9768-1332939bf333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:17.427599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1270/2555] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:17.431977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefc1fa-ab80-41f6-9233-e4ff828272bb None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:17.441951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefc1fa-ab80-41f6-9233-e4ff828272bb None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:17.446581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a88d5-b800-4b80-950c-8b0a3da28f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:17.447174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a88d5-b800-4b80-950c-8b0a3da28f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.447461 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a88d5-b800-4b80-950c-8b0a3da28f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.447723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a88d5-b800-4b80-950c-8b0a3da28f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.448399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a88d5-b800-4b80-950c-8b0a3da28f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:29:17.448954 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a88d5-b800-4b80-950c-8b0a3da28f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:17.451047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a88d5-b800-4b80-950c-8b0a3da28f16 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:17.462758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1286/2556] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:29:17.467691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bbc32-9a12-4445-9768-1332939bf333 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:17.476938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bbc32-9a12-4445-9768-1332939bf333 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:17.481634 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85574fb5-6b36-498b-9503-9c39c77bdca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:17.482228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85574fb5-6b36-498b-9503-9c39c77bdca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.482507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85574fb5-6b36-498b-9503-9c39c77bdca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.482780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85574fb5-6b36-498b-9503-9c39c77bdca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.483401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85574fb5-6b36-498b-9503-9c39c77bdca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:17.485336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85574fb5-6b36-498b-9503-9c39c77bdca1 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:17.513825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1271/2557] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:17 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:17.518030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a88d5-b800-4b80-950c-8b0a3da28f16 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:17.528389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a88d5-b800-4b80-950c-8b0a3da28f16 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:17.587264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40185397-4625-415a-822f-74237f6c9b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:17.587823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40185397-4625-415a-822f-74237f6c9b90 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.588113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40185397-4625-415a-822f-74237f6c9b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.588391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40185397-4625-415a-822f-74237f6c9b90 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.589029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40185397-4625-415a-822f-74237f6c9b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:17.591009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40185397-4625-415a-822f-74237f6c9b90 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:17.621232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1287/2558] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:29:17 2022] POST /identity/v3/users => generated 305 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:17.625844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85574fb5-6b36-498b-9503-9c39c77bdca1 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:17.636547 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85574fb5-6b36-498b-9503-9c39c77bdca1 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:17.641980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c4ea7-2ee6-4f2a-8abc-326df165498a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:17.642587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4ea7-2ee6-4f2a-8abc-326df165498a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.642866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4ea7-2ee6-4f2a-8abc-326df165498a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.643136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4ea7-2ee6-4f2a-8abc-326df165498a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.644189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c4ea7-2ee6-4f2a-8abc-326df165498a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:17.645401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c4ea7-2ee6-4f2a-8abc-326df165498a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:17.656267 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1272/2559] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:29:17.660313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40185397-4625-415a-822f-74237f6c9b90 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:17.665689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:17.671626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40185397-4625-415a-822f-74237f6c9b90 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:17.678648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05656a8d-7ae4-417b-b771-de5342bcd423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:17.679286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05656a8d-7ae4-417b-b771-de5342bcd423 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.679567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05656a8d-7ae4-417b-b771-de5342bcd423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.679871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05656a8d-7ae4-417b-b771-de5342bcd423 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.680564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05656a8d-7ae4-417b-b771-de5342bcd423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:17.682196 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05656a8d-7ae4-417b-b771-de5342bcd423 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:17.697248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1288/2560] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:17.698676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c4ea7-2ee6-4f2a-8abc-326df165498a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:17.709392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c4ea7-2ee6-4f2a-8abc-326df165498a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:17.713950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d134e15-31bf-4241-9874-272422a4267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:17.714588 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d134e15-31bf-4241-9874-272422a4267f admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.714863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d134e15-31bf-4241-9874-272422a4267f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.715131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d134e15-31bf-4241-9874-272422a4267f admin admin] PATH_INFO: `/v3/projects/89f119d38eae405b96103689d4036f6d/users/012e99c1165e46d98b96f1b272064afc/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.729808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d134e15-31bf-4241-9874-272422a4267f admin admin] RBAC: Authorizing `identity:create_grant(project_id=89f119d38eae405b96103689d4036f6d, user_id=012e99c1165e46d98b96f1b272064afc, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:17.732533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d134e15-31bf-4241-9874-272422a4267f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:17.748262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1273/2561] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:17 2022] PUT /identity/v3/projects/89f119d38eae405b96103689d4036f6d/users/012e99c1165e46d98b96f1b272064afc/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:29:17.759459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b74a7-549d-4f41-a7a9-54848af8fa1b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.759802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b74a7-549d-4f41-a7a9-54848af8fa1b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.760030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b74a7-549d-4f41-a7a9-54848af8fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.793205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d93b74a7-549d-4f41-a7a9-54848af8fa1b None None] MFA Rules not processed for user `012e99c1165e46d98b96f1b272064afc`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:17.794570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d93b74a7-549d-4f41-a7a9-54848af8fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:17.867848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1289/2562] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:17 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:17.873515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d134e15-31bf-4241-9874-272422a4267f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:17.936192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d134e15-31bf-4241-9874-272422a4267f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:17.940826 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882131a-c02e-4d15-ab4e-8804a6d25a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:17.941418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882131a-c02e-4d15-ab4e-8804a6d25a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:17.941693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882131a-c02e-4d15-ab4e-8804a6d25a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:17.941961 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882131a-c02e-4d15-ab4e-8804a6d25a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:17.942578 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882131a-c02e-4d15-ab4e-8804a6d25a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:17.944504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882131a-c02e-4d15-ab4e-8804a6d25a28 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:17.971155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1274/2563] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:17 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:17.975278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b74a7-549d-4f41-a7a9-54848af8fa1b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:17.986312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93b74a7-549d-4f41-a7a9-54848af8fa1b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:18.041477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b29f47-2a57-461e-a220-e925874b7ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:18.042086 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b29f47-2a57-461e-a220-e925874b7ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:18.042371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b29f47-2a57-461e-a220-e925874b7ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:18.042647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b29f47-2a57-461e-a220-e925874b7ea5 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:18.043279 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b29f47-2a57-461e-a220-e925874b7ea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:18.045838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b29f47-2a57-461e-a220-e925874b7ea5 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:18.067238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1290/2564] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:29:17 2022] POST /identity/v3/users => generated 311 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:18.071130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882131a-c02e-4d15-ab4e-8804a6d25a28 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:18.081719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882131a-c02e-4d15-ab4e-8804a6d25a28 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:18.086353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69d447-b035-4290-baed-738a68fdd66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:18.086947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69d447-b035-4290-baed-738a68fdd66e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:18.087241 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69d447-b035-4290-baed-738a68fdd66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:18.087508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69d447-b035-4290-baed-738a68fdd66e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:18.088176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69d447-b035-4290-baed-738a68fdd66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:18.089775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69d447-b035-4290-baed-738a68fdd66e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:18.100245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1275/2565] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:18.104287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b29f47-2a57-461e-a220-e925874b7ea5 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:18.114765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b29f47-2a57-461e-a220-e925874b7ea5 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:18.119341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbb21e-ddce-46f3-b6eb-6e4e8bf4314f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:18.120017 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbb21e-ddce-46f3-b6eb-6e4e8bf4314f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:18.120315 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbb21e-ddce-46f3-b6eb-6e4e8bf4314f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:18.120584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbb21e-ddce-46f3-b6eb-6e4e8bf4314f admin admin] PATH_INFO: `/v3/projects/7c08e6b4189747099e26be485f277a65/users/f96017fe03494f9da26d32edc46c5e1a/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:18.136109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbb21e-ddce-46f3-b6eb-6e4e8bf4314f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c08e6b4189747099e26be485f277a65, user_id=f96017fe03494f9da26d32edc46c5e1a, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:18.138826 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbb21e-ddce-46f3-b6eb-6e4e8bf4314f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:18.151425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1291/2566] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:18 2022] PUT /identity/v3/projects/7c08e6b4189747099e26be485f277a65/users/f96017fe03494f9da26d32edc46c5e1a/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:18.155621 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69d447-b035-4290-baed-738a68fdd66e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:18.165798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69d447-b035-4290-baed-738a68fdd66e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:18.186447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:18.225264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0f173-de01-4bad-9fb7-4d8d9cc3df57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:18.225869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0f173-de01-4bad-9fb7-4d8d9cc3df57 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:18.226149 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0f173-de01-4bad-9fb7-4d8d9cc3df57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:18.226423 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0f173-de01-4bad-9fb7-4d8d9cc3df57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:18.227055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0f173-de01-4bad-9fb7-4d8d9cc3df57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:18.228671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0f173-de01-4bad-9fb7-4d8d9cc3df57 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:18.238834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1276/2567] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:18.242993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbb21e-ddce-46f3-b6eb-6e4e8bf4314f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:18.253871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbb21e-ddce-46f3-b6eb-6e4e8bf4314f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:18.258529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6940c-7c37-46b8-8bb8-6d37254f341a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:18.259176 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6940c-7c37-46b8-8bb8-6d37254f341a admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:18.259459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6940c-7c37-46b8-8bb8-6d37254f341a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:18.259734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6940c-7c37-46b8-8bb8-6d37254f341a admin admin] PATH_INFO: `/v3/projects/7c08e6b4189747099e26be485f277a65/users/f96017fe03494f9da26d32edc46c5e1a/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:18.261697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6940c-7c37-46b8-8bb8-6d37254f341a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c08e6b4189747099e26be485f277a65, user_id=f96017fe03494f9da26d32edc46c5e1a, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:18.264385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6940c-7c37-46b8-8bb8-6d37254f341a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:18.274453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1292/2568] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:18 2022] PUT /identity/v3/projects/7c08e6b4189747099e26be485f277a65/users/f96017fe03494f9da26d32edc46c5e1a/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:18.285153 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49d6d2-e8cb-4b31-bf45-47c24d2c6a54 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:18.285531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49d6d2-e8cb-4b31-bf45-47c24d2c6a54 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:18.285882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49d6d2-e8cb-4b31-bf45-47c24d2c6a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:18.316313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2b49d6d2-e8cb-4b31-bf45-47c24d2c6a54 None None] MFA Rules not processed for user `f96017fe03494f9da26d32edc46c5e1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:18.317569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2b49d6d2-e8cb-4b31-bf45-47c24d2c6a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:18.389448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1277/2569] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:18.400182 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6940c-7c37-46b8-8bb8-6d37254f341a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:18.401976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9b6940c-7c37-46b8-8bb8-6d37254f341a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:18.438823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:18.462230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6940c-7c37-46b8-8bb8-6d37254f341a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:18.466476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062d2bd-5530-47a4-9c83-c9840ffe8391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:18.467080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d2bd-5530-47a4-9c83-c9840ffe8391 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:18.467361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d2bd-5530-47a4-9c83-c9840ffe8391 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:18.467634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062d2bd-5530-47a4-9c83-c9840ffe8391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:18.476654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062d2bd-5530-47a4-9c83-c9840ffe8391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:18.478077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062d2bd-5530-47a4-9c83-c9840ffe8391 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:18.488075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1293/2570] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:18 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:18.641710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49d6d2-e8cb-4b31-bf45-47c24d2c6a54 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:18.651595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49d6d2-e8cb-4b31-bf45-47c24d2c6a54 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:18.655929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe2a86-7658-4644-a028-823d123e0006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:18.656530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe2a86-7658-4644-a028-823d123e0006 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:18.656811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe2a86-7658-4644-a028-823d123e0006 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:18.657083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe2a86-7658-4644-a028-823d123e0006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:18.713694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe2a86-7658-4644-a028-823d123e0006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:18.715220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe2a86-7658-4644-a028-823d123e0006 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:18.725406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1278/2571] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:18 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:19.148575 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062d2bd-5530-47a4-9c83-c9840ffe8391 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:19.150068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1062d2bd-5530-47a4-9c83-c9840ffe8391 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:19.211036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062d2bd-5530-47a4-9c83-c9840ffe8391 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:19.214609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac820cf-5692-421e-bd07-bb0ee2036d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:19.215204 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac820cf-5692-421e-bd07-bb0ee2036d74 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:19.215480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac820cf-5692-421e-bd07-bb0ee2036d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:19.215746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac820cf-5692-421e-bd07-bb0ee2036d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:19.217473 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9ac820cf-5692-421e-bd07-bb0ee2036d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:19.279822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac820cf-5692-421e-bd07-bb0ee2036d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:19.281260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac820cf-5692-421e-bd07-bb0ee2036d74 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:19.290684 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1294/2572] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:28.216008 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe2a86-7658-4644-a028-823d123e0006 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:28.260995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:28.279587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe2a86-7658-4644-a028-823d123e0006 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:28.284104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:28.284735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:28.285015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:28.285289 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] PATH_INFO: `/v3/users/012e99c1165e46d98b96f1b272064afc` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:28.286426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=012e99c1165e46d98b96f1b272064afc)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:28.288400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:28.334482 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '012e99c1165e46d98b96f1b272064afc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.335124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '012e99c1165e46d98b96f1b272064afc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.342063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '012e99c1165e46d98b96f1b272064afc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.342778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '012e99c1165e46d98b96f1b272064afc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.349664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '012e99c1165e46d98b96f1b272064afc'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.367251 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1279/2573] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:29:28 2022] DELETE /identity/v3/users/012e99c1165e46d98b96f1b272064afc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:28.372609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5744a0-cd0d-42b1-990a-8c23890bb07c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:28.373091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5744a0-cd0d-42b1-990a-8c23890bb07c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:28.373195 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5744a0-cd0d-42b1-990a-8c23890bb07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:28.391780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5c5744a0-cd0d-42b1-990a-8c23890bb07c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:28.393004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c5744a0-cd0d-42b1-990a-8c23890bb07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:28.451805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1295/2574] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:28.461524 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:28.462963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:28.521890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abe87e-36c9-4bcb-84ff-baf91a9f63cb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:28.525630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b4a92-374e-4c60-a17d-a07651fed973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:28.526233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4a92-374e-4c60-a17d-a07651fed973 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:28.526515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4a92-374e-4c60-a17d-a07651fed973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:28.526790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b4a92-374e-4c60-a17d-a07651fed973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:28.536328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b4a92-374e-4c60-a17d-a07651fed973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:28.537765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b4a92-374e-4c60-a17d-a07651fed973 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:28.548130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1280/2575] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:28.880704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5744a0-cd0d-42b1-990a-8c23890bb07c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:28.882149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5c5744a0-cd0d-42b1-990a-8c23890bb07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:28.895435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5744a0-cd0d-42b1-990a-8c23890bb07c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:28.900358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:28.901024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:28.901309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:28.901765 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] PATH_INFO: `/v3/projects/89f119d38eae405b96103689d4036f6d` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:28.903467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89f119d38eae405b96103689d4036f6d)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:28.908986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:28.933888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=100 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:28.958590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89f119d38eae405b96103689d4036f6d', 'request_id': 'req-0541f3aa-5466-4859-99ae-b0ee4f8eff65'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.964822 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89f119d38eae405b96103689d4036f6d', 'request_id': 'req-0541f3aa-5466-4859-99ae-b0ee4f8eff65'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.973007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89f119d38eae405b96103689d4036f6d', 'request_id': 'req-0541f3aa-5466-4859-99ae-b0ee4f8eff65'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.984277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] The token cache is being invalidate because project 89f119d38eae405b96103689d4036f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:28.984611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.985295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:28.986870 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1296/2576] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:28 2022] DELETE /identity/v3/projects/89f119d38eae405b96103689d4036f6d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:28.990904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b4a92-374e-4c60-a17d-a07651fed973 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:29.000351 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b4a92-374e-4c60-a17d-a07651fed973 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:29.055781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:29.056419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.056691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.057073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] PATH_INFO: `/v3/users/f96017fe03494f9da26d32edc46c5e1a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.058235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96017fe03494f9da26d32edc46c5e1a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:29.060129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:29.065606 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:29.108797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96017fe03494f9da26d32edc46c5e1a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.109442 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96017fe03494f9da26d32edc46c5e1a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.116975 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96017fe03494f9da26d32edc46c5e1a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.117655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96017fe03494f9da26d32edc46c5e1a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.124760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-54d7b594-7a67-4f6f-b5f4-ec81592ca543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96017fe03494f9da26d32edc46c5e1a'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.141860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1281/2577] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:29:28 2022] DELETE /identity/v3/users/f96017fe03494f9da26d32edc46c5e1a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:29.410025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:29.411558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:29.425512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541f3aa-5466-4859-99ae-b0ee4f8eff65 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:29.484640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:29.485274 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.485555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.485848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] PATH_INFO: `/v3/projects/7c08e6b4189747099e26be485f277a65` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.487004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c08e6b4189747099e26be485f277a65)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:29.488974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:29.534878 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c08e6b4189747099e26be485f277a65', 'request_id': 'req-e7c42da8-48d3-4a69-953f-3d4390bad6e4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.542201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c08e6b4189747099e26be485f277a65', 'request_id': 'req-e7c42da8-48d3-4a69-953f-3d4390bad6e4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.550694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c08e6b4189747099e26be485f277a65', 'request_id': 'req-e7c42da8-48d3-4a69-953f-3d4390bad6e4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.561758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] The token cache is being invalidate because project 7c08e6b4189747099e26be485f277a65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:29.562176 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.562815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e7c42da8-48d3-4a69-953f-3d4390bad6e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:29.564348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1297/2578] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:29 2022] DELETE /identity/v3/projects/7c08e6b4189747099e26be485f277a65 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:29.569383 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0a1b-042e-408a-b4d2-b4a3ed7e9d33 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.569869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0a1b-042e-408a-b4d2-b4a3ed7e9d33 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.569966 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc0a1b-042e-408a-b4d2-b4a3ed7e9d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.590642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c1cc0a1b-042e-408a-b4d2-b4a3ed7e9d33 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:29.591923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c1cc0a1b-042e-408a-b4d2-b4a3ed7e9d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:29.651732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1282/2579] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:29.659650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88e071-1a42-4aed-b750-2b49b3969936 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.659928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88e071-1a42-4aed-b750-2b49b3969936 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.660212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88e071-1a42-4aed-b750-2b49b3969936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.679681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7f88e071-1a42-4aed-b750-2b49b3969936 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:29.680925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7f88e071-1a42-4aed-b750-2b49b3969936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:29.689437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1298/2580] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:29.693630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc0a1b-042e-408a-b4d2-b4a3ed7e9d33 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:29.704119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc0a1b-042e-408a-b4d2-b4a3ed7e9d33 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:29.708648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7ea74-3138-42f6-ae70-07d5692d423b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:29.709249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7ea74-3138-42f6-ae70-07d5692d423b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.709529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7ea74-3138-42f6-ae70-07d5692d423b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.709795 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7ea74-3138-42f6-ae70-07d5692d423b admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.710443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7ea74-3138-42f6-ae70-07d5692d423b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:29:29.710737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7ea74-3138-42f6-ae70-07d5692d423b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:29.712375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba7ea74-3138-42f6-ae70-07d5692d423b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:29.727448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1283/2581] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:29:29.728580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88e071-1a42-4aed-b750-2b49b3969936 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:29.737763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88e071-1a42-4aed-b750-2b49b3969936 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:29.742815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114a160-f599-4cc9-a29d-a4f7a278e0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:29.743430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a160-f599-4cc9-a29d-a4f7a278e0ea admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.743709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a160-f599-4cc9-a29d-a4f7a278e0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.744015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9114a160-f599-4cc9-a29d-a4f7a278e0ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.744647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114a160-f599-4cc9-a29d-a4f7a278e0ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:29.747057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9114a160-f599-4cc9-a29d-a4f7a278e0ea admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:29.779660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1299/2582] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:29 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:29.783846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7ea74-3138-42f6-ae70-07d5692d423b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:29.794140 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7ea74-3138-42f6-ae70-07d5692d423b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:29.855015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66e18f-3464-4dbf-a7f5-6cea78b4eaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:29.855631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66e18f-3464-4dbf-a7f5-6cea78b4eaa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.856479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66e18f-3464-4dbf-a7f5-6cea78b4eaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.856838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66e18f-3464-4dbf-a7f5-6cea78b4eaa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.857942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66e18f-3464-4dbf-a7f5-6cea78b4eaa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:29.861622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66e18f-3464-4dbf-a7f5-6cea78b4eaa2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:29.889651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1284/2583] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:29:29 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:29:29.890875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114a160-f599-4cc9-a29d-a4f7a278e0ea admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:29.901373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9114a160-f599-4cc9-a29d-a4f7a278e0ea admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:29.906184 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db270397-baf9-4bd2-9f00-669273cb6ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:29.906780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db270397-baf9-4bd2-9f00-669273cb6ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.907061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db270397-baf9-4bd2-9f00-669273cb6ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.907336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db270397-baf9-4bd2-9f00-669273cb6ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.908012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db270397-baf9-4bd2-9f00-669273cb6ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:29.909601 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db270397-baf9-4bd2-9f00-669273cb6ce6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:29.914885 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:29.922192 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1300/2584] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:29.926630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66e18f-3464-4dbf-a7f5-6cea78b4eaa2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:29.932130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:29.939522 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66e18f-3464-4dbf-a7f5-6cea78b4eaa2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:29.945222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f67753-7af2-4c8f-85b2-04512bb15156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:29.945708 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f67753-7af2-4c8f-85b2-04512bb15156 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.945988 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f67753-7af2-4c8f-85b2-04512bb15156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.946252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f67753-7af2-4c8f-85b2-04512bb15156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.946882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f67753-7af2-4c8f-85b2-04512bb15156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:29.948765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f67753-7af2-4c8f-85b2-04512bb15156 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:29.959501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1285/2585] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:29.963701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db270397-baf9-4bd2-9f00-669273cb6ce6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:29.974019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db270397-baf9-4bd2-9f00-669273cb6ce6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:29.978679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ef1ca-3667-4ddf-b1db-ba9a51c17ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:29.979327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ef1ca-3667-4ddf-b1db-ba9a51c17ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:29.979608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ef1ca-3667-4ddf-b1db-ba9a51c17ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:29.979924 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ef1ca-3667-4ddf-b1db-ba9a51c17ac1 admin admin] PATH_INFO: `/v3/projects/384c442c805c43d78e94bc5719976a09/users/85e74d8f9e0a4dddb44bd239272da8a1/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:29.995006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ef1ca-3667-4ddf-b1db-ba9a51c17ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=384c442c805c43d78e94bc5719976a09, user_id=85e74d8f9e0a4dddb44bd239272da8a1, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:29.998418 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ef1ca-3667-4ddf-b1db-ba9a51c17ac1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:30.011135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1301/2586] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:29 2022] PUT /identity/v3/projects/384c442c805c43d78e94bc5719976a09/users/85e74d8f9e0a4dddb44bd239272da8a1/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:30.016216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebc568-e1a3-42c1-8a7e-607c5e19f61e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:30.016503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebc568-e1a3-42c1-8a7e-607c5e19f61e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:30.016776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebc568-e1a3-42c1-8a7e-607c5e19f61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:30.035210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-89ebc568-e1a3-42c1-8a7e-607c5e19f61e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:30.036462 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-89ebc568-e1a3-42c1-8a7e-607c5e19f61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:30.104476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1286/2587] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:29:30.114704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ef1ca-3667-4ddf-b1db-ba9a51c17ac1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:30.116440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9f5ef1ca-3667-4ddf-b1db-ba9a51c17ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:30.178326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ef1ca-3667-4ddf-b1db-ba9a51c17ac1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:30.181911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b556cc3-607a-44f7-bc0b-6675429f63fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:30.182506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b556cc3-607a-44f7-bc0b-6675429f63fe service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:30.182784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b556cc3-607a-44f7-bc0b-6675429f63fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:30.183052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b556cc3-607a-44f7-bc0b-6675429f63fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:30.192449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b556cc3-607a-44f7-bc0b-6675429f63fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:30.193869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b556cc3-607a-44f7-bc0b-6675429f63fe service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:30.203394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1302/2588] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:30.919748 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0af63-919e-4189-8e65-f489979fd5dc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:30.920307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0af63-919e-4189-8e65-f489979fd5dc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:30.920307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0af63-919e-4189-8e65-f489979fd5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:30.953485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-04f0af63-919e-4189-8e65-f489979fd5dc None None] MFA Rules not processed for user `85e74d8f9e0a4dddb44bd239272da8a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:30.955542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-04f0af63-919e-4189-8e65-f489979fd5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:31.026821 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bee46a-d8af-4ab9-b707-efd5bfe7179a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:31.027373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bee46a-d8af-4ab9-b707-efd5bfe7179a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:31.027373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bee46a-d8af-4ab9-b707-efd5bfe7179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:31.044868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1287/2589] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:29:31.049718 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0af63-919e-4189-8e65-f489979fd5dc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:31.061643 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f6bee46a-d8af-4ab9-b707-efd5bfe7179a None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:31.063586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f6bee46a-d8af-4ab9-b707-efd5bfe7179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:31.066494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0af63-919e-4189-8e65-f489979fd5dc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:31.075805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce46b8-b8b6-4610-ab0d-d8be89d583f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:31.082454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce46b8-b8b6-4610-ab0d-d8be89d583f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:31.082787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce46b8-b8b6-4610-ab0d-d8be89d583f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:31.083068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce46b8-b8b6-4610-ab0d-d8be89d583f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:31.083737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce46b8-b8b6-4610-ab0d-d8be89d583f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:31.085876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce46b8-b8b6-4610-ab0d-d8be89d583f4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:31.125183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1288/2590] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:31 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:31.129105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce46b8-b8b6-4610-ab0d-d8be89d583f4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:31.139215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1303/2591] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:29:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:29:31.190939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bee46a-d8af-4ab9-b707-efd5bfe7179a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:31.192364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f6bee46a-d8af-4ab9-b707-efd5bfe7179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:31.200273 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce46b8-b8b6-4610-ab0d-d8be89d583f4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:31.205620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f6d54-fa17-4baa-a1fe-90a28c92490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:31.206310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f6d54-fa17-4baa-a1fe-90a28c92490c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:31.206596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f6d54-fa17-4baa-a1fe-90a28c92490c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:31.206872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f6d54-fa17-4baa-a1fe-90a28c92490c admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:31.207508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f6d54-fa17-4baa-a1fe-90a28c92490c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:31.209444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f6d54-fa17-4baa-a1fe-90a28c92490c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:31.217614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:31.231607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1289/2592] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:29:31 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:29:31.235677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f6d54-fa17-4baa-a1fe-90a28c92490c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:31.245250 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f6d54-fa17-4baa-a1fe-90a28c92490c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:31.250610 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aafce-699d-4345-917c-84db3cabcd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:31.251199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aafce-699d-4345-917c-84db3cabcd4b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:31.251478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aafce-699d-4345-917c-84db3cabcd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:31.251806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aafce-699d-4345-917c-84db3cabcd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:31.252443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aafce-699d-4345-917c-84db3cabcd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:31.254028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835aafce-699d-4345-917c-84db3cabcd4b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:31.264737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1290/2593] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:29:31.266866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bee46a-d8af-4ab9-b707-efd5bfe7179a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:31.268652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aafce-699d-4345-917c-84db3cabcd4b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:31.275733 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:31.278498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aafce-699d-4345-917c-84db3cabcd4b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:31.283318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd146a-5814-417e-a6c8-9b0997592873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:31.284030 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd146a-5814-417e-a6c8-9b0997592873 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:31.284301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd146a-5814-417e-a6c8-9b0997592873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:31.284571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd146a-5814-417e-a6c8-9b0997592873 admin admin] PATH_INFO: `/v3/projects/1a8ac0fe91d94a8185729bc9c047969d/users/48652bfd024547d6824f4bd49b96098f/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:31.299720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd146a-5814-417e-a6c8-9b0997592873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8ac0fe91d94a8185729bc9c047969d, user_id=48652bfd024547d6824f4bd49b96098f, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:31.302414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd146a-5814-417e-a6c8-9b0997592873 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:31.317003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1291/2594] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:31 2022] PUT /identity/v3/projects/1a8ac0fe91d94a8185729bc9c047969d/users/48652bfd024547d6824f4bd49b96098f/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:29:31.320952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd146a-5814-417e-a6c8-9b0997592873 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:31.333794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9590af-e94a-4266-b7a5-a3438156fafc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:31.334425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9590af-e94a-4266-b7a5-a3438156fafc service swift] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:31.334764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9590af-e94a-4266-b7a5-a3438156fafc service swift] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:31.335122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9590af-e94a-4266-b7a5-a3438156fafc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:31.389998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd146a-5814-417e-a6c8-9b0997592873 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:31.393236 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9590af-e94a-4266-b7a5-a3438156fafc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:31.394787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a276ba-bea0-43c2-918e-e3de627cafba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:31.394968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9590af-e94a-4266-b7a5-a3438156fafc service swift] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:31.395411 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a276ba-bea0-43c2-918e-e3de627cafba admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:31.395695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a276ba-bea0-43c2-918e-e3de627cafba admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:31.396039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a276ba-bea0-43c2-918e-e3de627cafba admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:31.396777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a276ba-bea0-43c2-918e-e3de627cafba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:31.398971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a276ba-bea0-43c2-918e-e3de627cafba admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:31.403627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1304/2595] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:29:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 12:29:31.411786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1292/2596] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:29:31.415034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9590af-e94a-4266-b7a5-a3438156fafc service swift] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:31.425324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9590af-e94a-4266-b7a5-a3438156fafc service swift] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:31.439135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e2139-690d-43f0-b5b2-637a884b1847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:31.439964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e2139-690d-43f0-b5b2-637a884b1847 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:31.440282 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e2139-690d-43f0-b5b2-637a884b1847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:31.440550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e2139-690d-43f0-b5b2-637a884b1847 admin admin] PATH_INFO: `/v3/projects/1a8ac0fe91d94a8185729bc9c047969d/users/48652bfd024547d6824f4bd49b96098f/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:31.443437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e2139-690d-43f0-b5b2-637a884b1847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8ac0fe91d94a8185729bc9c047969d, user_id=48652bfd024547d6824f4bd49b96098f, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:31.446895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e2139-690d-43f0-b5b2-637a884b1847 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:31.463233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1305/2597] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:31 2022] PUT /identity/v3/projects/1a8ac0fe91d94a8185729bc9c047969d/users/48652bfd024547d6824f4bd49b96098f/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:32.447104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed3ae6-145f-4d51-b8d6-422e4553b87c None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:32.447723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed3ae6-145f-4d51-b8d6-422e4553b87c None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:32.447723 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed3ae6-145f-4d51-b8d6-422e4553b87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:32.478746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-90ed3ae6-145f-4d51-b8d6-422e4553b87c None None] MFA Rules not processed for user `48652bfd024547d6824f4bd49b96098f`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:32.480367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-90ed3ae6-145f-4d51-b8d6-422e4553b87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:32.584786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1293/2598] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:32.585593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e2139-690d-43f0-b5b2-637a884b1847 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:32.587280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bc5e2139-690d-43f0-b5b2-637a884b1847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:32.652434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e2139-690d-43f0-b5b2-637a884b1847 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:32.656845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ac219-b1ad-41c1-9a28-e466e978b925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:32.657459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ac219-b1ad-41c1-9a28-e466e978b925 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:32.657739 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ac219-b1ad-41c1-9a28-e466e978b925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:32.658154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ac219-b1ad-41c1-9a28-e466e978b925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:32.723052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:32.727578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ac219-b1ad-41c1-9a28-e466e978b925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:32.729708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ac219-b1ad-41c1-9a28-e466e978b925 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:32.746097 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1306/2599] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:34.200520 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed3ae6-145f-4d51-b8d6-422e4553b87c None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:34.202041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-90ed3ae6-145f-4d51-b8d6-422e4553b87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:34.214385 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed3ae6-145f-4d51-b8d6-422e4553b87c None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:34.218657 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750d318-945e-4ddb-a3a0-e100f16cf481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:34.219272 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750d318-945e-4ddb-a3a0-e100f16cf481 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:34.219564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750d318-945e-4ddb-a3a0-e100f16cf481 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:34.219846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750d318-945e-4ddb-a3a0-e100f16cf481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:34.228893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750d318-945e-4ddb-a3a0-e100f16cf481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:34.230407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750d318-945e-4ddb-a3a0-e100f16cf481 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:34.240937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1294/2600] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:42.736452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:44.236554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=4,keystone:SELECT=86 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:48.326609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ac219-b1ad-41c1-9a28-e466e978b925 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:48.395663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ac219-b1ad-41c1-9a28-e466e978b925 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:48.400834 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:48.401493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:48.402452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:48.402726 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] PATH_INFO: `/v3/users/85e74d8f9e0a4dddb44bd239272da8a1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:48.403892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85e74d8f9e0a4dddb44bd239272da8a1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:48.405848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:48.463055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85e74d8f9e0a4dddb44bd239272da8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.473217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85e74d8f9e0a4dddb44bd239272da8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.474039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85e74d8f9e0a4dddb44bd239272da8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.476290 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85e74d8f9e0a4dddb44bd239272da8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.489430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85e74d8f9e0a4dddb44bd239272da8a1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.512783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1307/2601] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:29:48 2022] DELETE /identity/v3/users/85e74d8f9e0a4dddb44bd239272da8a1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:48.702834 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750d318-945e-4ddb-a3a0-e100f16cf481 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:48.717843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750d318-945e-4ddb-a3a0-e100f16cf481 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:48.781938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:48.782587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:48.782869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:48.783145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] PATH_INFO: `/v3/projects/384c442c805c43d78e94bc5719976a09` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:48.784389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=384c442c805c43d78e94bc5719976a09)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:48.786310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:48.835215 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384c442c805c43d78e94bc5719976a09', 'request_id': 'req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.842677 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384c442c805c43d78e94bc5719976a09', 'request_id': 'req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.851485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384c442c805c43d78e94bc5719976a09', 'request_id': 'req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.863012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] The token cache is being invalidate because project 384c442c805c43d78e94bc5719976a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:48.863320 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.863982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:48.865486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1295/2602] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:48 2022] DELETE /identity/v3/projects/384c442c805c43d78e94bc5719976a09 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:48.869536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:48.870997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:48.884288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f9fd9-7193-410a-bf23-d969ad6b6fa9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:48.944146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:48.944777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:48.945066 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:48.945337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] PATH_INFO: `/v3/users/48652bfd024547d6824f4bd49b96098f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:48.946523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] RBAC: Authorizing `identity:delete_user(user_id=48652bfd024547d6824f4bd49b96098f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:48.949230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:48.974399 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:49.001825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48652bfd024547d6824f4bd49b96098f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.009340 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48652bfd024547d6824f4bd49b96098f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.010025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48652bfd024547d6824f4bd49b96098f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.010772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48652bfd024547d6824f4bd49b96098f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.018314 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-60b4d7f4-f99e-4cbe-8f4f-835d493c003f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48652bfd024547d6824f4bd49b96098f'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.035155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1308/2603] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:29:48 2022] DELETE /identity/v3/users/48652bfd024547d6824f4bd49b96098f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:49.217254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:49.218768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:49.231234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb6e71-5e58-4514-8163-b395c9b2bb3e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:49.302449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:49.303194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.303543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.303925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] PATH_INFO: `/v3/projects/1a8ac0fe91d94a8185729bc9c047969d` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.305366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a8ac0fe91d94a8185729bc9c047969d)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:49.307303 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:49.348706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:49.367711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a8ac0fe91d94a8185729bc9c047969d', 'request_id': 'req-813829b4-391e-4f33-975c-a82e1abfd945'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.376839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a8ac0fe91d94a8185729bc9c047969d', 'request_id': 'req-813829b4-391e-4f33-975c-a82e1abfd945'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.386582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a8ac0fe91d94a8185729bc9c047969d', 'request_id': 'req-813829b4-391e-4f33-975c-a82e1abfd945'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.406920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] The token cache is being invalidate because project 1a8ac0fe91d94a8185729bc9c047969d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:29:49.407244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.407938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-813829b4-391e-4f33-975c-a82e1abfd945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:29:49.414426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1296/2604] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:29:49 2022] DELETE /identity/v3/projects/1a8ac0fe91d94a8185729bc9c047969d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:49.416176 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa3612-37ad-4f03-a3a2-fdc019686793 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.416658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa3612-37ad-4f03-a3a2-fdc019686793 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.416773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa3612-37ad-4f03-a3a2-fdc019686793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.437992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c0aa3612-37ad-4f03-a3a2-fdc019686793 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:49.439334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c0aa3612-37ad-4f03-a3a2-fdc019686793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:49.507894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1309/2605] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:49.517065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f292ded-6e9a-4de9-90a7-94c5e560319d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.517347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f292ded-6e9a-4de9-90a7-94c5e560319d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.517618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f292ded-6e9a-4de9-90a7-94c5e560319d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.537625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-2f292ded-6e9a-4de9-90a7-94c5e560319d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:49.538848 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-2f292ded-6e9a-4de9-90a7-94c5e560319d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:49.547500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1297/2606] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:49.551915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa3612-37ad-4f03-a3a2-fdc019686793 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:49.562811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa3612-37ad-4f03-a3a2-fdc019686793 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:49.567482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a02f7-a8d0-4f6a-bd6b-a3cd8a2ad831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:49.568111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a02f7-a8d0-4f6a-bd6b-a3cd8a2ad831 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.568386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a02f7-a8d0-4f6a-bd6b-a3cd8a2ad831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.568679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a02f7-a8d0-4f6a-bd6b-a3cd8a2ad831 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.569321 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a02f7-a8d0-4f6a-bd6b-a3cd8a2ad831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:29:49.569621 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a02f7-a8d0-4f6a-bd6b-a3cd8a2ad831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:49.571214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a02f7-a8d0-4f6a-bd6b-a3cd8a2ad831 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:49.584198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1310/2607] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:29:49.588658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f292ded-6e9a-4de9-90a7-94c5e560319d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:49.598245 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f292ded-6e9a-4de9-90a7-94c5e560319d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:49.603503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289e597-9113-4f87-9cf2-007b00e58886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:49.604116 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289e597-9113-4f87-9cf2-007b00e58886 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.604394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289e597-9113-4f87-9cf2-007b00e58886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.605359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289e597-9113-4f87-9cf2-007b00e58886 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.605918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289e597-9113-4f87-9cf2-007b00e58886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:49.607777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289e597-9113-4f87-9cf2-007b00e58886 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:49.641060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1298/2608] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:29:49 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:29:49.642343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a02f7-a8d0-4f6a-bd6b-a3cd8a2ad831 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:49.652952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a02f7-a8d0-4f6a-bd6b-a3cd8a2ad831 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:49.710362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ab586-e74f-4e4d-b849-1671a223204d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:49.710893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ab586-e74f-4e4d-b849-1671a223204d admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.711180 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ab586-e74f-4e4d-b849-1671a223204d admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.711451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ab586-e74f-4e4d-b849-1671a223204d admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.712152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ab586-e74f-4e4d-b849-1671a223204d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:49.714138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ab586-e74f-4e4d-b849-1671a223204d admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:49.737052 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1311/2609] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:29:49 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:29:49.741038 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289e597-9113-4f87-9cf2-007b00e58886 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:49.752404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289e597-9113-4f87-9cf2-007b00e58886 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:49.757246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d7199-a22d-49ce-af10-e4b4701552e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:49.757735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d7199-a22d-49ce-af10-e4b4701552e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.758043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d7199-a22d-49ce-af10-e4b4701552e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.758257 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d7199-a22d-49ce-af10-e4b4701552e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.758895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d7199-a22d-49ce-af10-e4b4701552e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:49.760731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d7199-a22d-49ce-af10-e4b4701552e6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:49.771381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1299/2610] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:49.775388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ab586-e74f-4e4d-b849-1671a223204d admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:49.785762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ab586-e74f-4e4d-b849-1671a223204d admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:49.790946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5196fc-8500-44a6-91e6-1c7207b71064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:49.791555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5196fc-8500-44a6-91e6-1c7207b71064 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.791875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5196fc-8500-44a6-91e6-1c7207b71064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.792215 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5196fc-8500-44a6-91e6-1c7207b71064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.793516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5196fc-8500-44a6-91e6-1c7207b71064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:49.795177 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5196fc-8500-44a6-91e6-1c7207b71064 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:49.809401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1312/2611] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:29:49.810871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d7199-a22d-49ce-af10-e4b4701552e6 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:49.821324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d7199-a22d-49ce-af10-e4b4701552e6 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:49.826342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83a5d9-d00d-4cff-b05f-6ed0583fce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:49.827061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83a5d9-d00d-4cff-b05f-6ed0583fce45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.827407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83a5d9-d00d-4cff-b05f-6ed0583fce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.827792 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83a5d9-d00d-4cff-b05f-6ed0583fce45 admin admin] PATH_INFO: `/v3/projects/05faadd85ad84c629070f527ba67a8df/users/91b2d8698d2f438ea57162d67f601e31/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.845014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83a5d9-d00d-4cff-b05f-6ed0583fce45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05faadd85ad84c629070f527ba67a8df, user_id=91b2d8698d2f438ea57162d67f601e31, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:49.847710 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83a5d9-d00d-4cff-b05f-6ed0583fce45 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:49.861840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1300/2612] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:29:49 2022] PUT /identity/v3/projects/05faadd85ad84c629070f527ba67a8df/users/91b2d8698d2f438ea57162d67f601e31/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:29:49.872396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993d21d-6283-4680-961a-19dfb7b6bb37 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:49.872686 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993d21d-6283-4680-961a-19dfb7b6bb37 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:49.872964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6993d21d-6283-4680-961a-19dfb7b6bb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:49.882082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:49.905401 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-6993d21d-6283-4680-961a-19dfb7b6bb37 None None] MFA Rules not processed for user `91b2d8698d2f438ea57162d67f601e31`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:49.906910 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6993d21d-6283-4680-961a-19dfb7b6bb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:49.971091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1313/2613] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:29:49.982484 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83a5d9-d00d-4cff-b05f-6ed0583fce45 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:49.984187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ad83a5d9-d00d-4cff-b05f-6ed0583fce45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:50.045296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83a5d9-d00d-4cff-b05f-6ed0583fce45 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:50.049235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fa34f-3f36-4882-b401-54953e8aceed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:50.049854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fa34f-3f36-4882-b401-54953e8aceed service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:50.050136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fa34f-3f36-4882-b401-54953e8aceed service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:50.050417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fa34f-3f36-4882-b401-54953e8aceed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:50.060043 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fa34f-3f36-4882-b401-54953e8aceed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:50.066825 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fa34f-3f36-4882-b401-54953e8aceed service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:50.077390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1301/2614] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:29:50.082078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f40875-350b-41b5-900a-4c5d58e71b9c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:50.082370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f40875-350b-41b5-900a-4c5d58e71b9c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:50.082653 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f40875-350b-41b5-900a-4c5d58e71b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:50.102546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-11f40875-350b-41b5-900a-4c5d58e71b9c None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:29:50.103787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-11f40875-350b-41b5-900a-4c5d58e71b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:50.165992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1314/2615] 10.222.0.71 () {58 vars in 1009 bytes} [Wed Sep 21 12:29:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:29:50.179024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fa34f-3f36-4882-b401-54953e8aceed service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:29:50.180489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0b8fa34f-3f36-4882-b401-54953e8aceed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:29:50.246942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:29:50.254927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fa34f-3f36-4882-b401-54953e8aceed service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:29:50.261891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e28f945-e670-4191-a41e-d6334ab29c3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:29:50.262504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28f945-e670-4191-a41e-d6334ab29c3c service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:29:50.262785 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28f945-e670-4191-a41e-d6334ab29c3c service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:29:50.263059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e28f945-e670-4191-a41e-d6334ab29c3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:29:50.272235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28f945-e670-4191-a41e-d6334ab29c3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:29:50.273861 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e28f945-e670-4191-a41e-d6334ab29c3c service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:29:50.286063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1302/2616] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:29:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 12:30:00.159132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=55 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:00.281908 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:00.644771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f40875-350b-41b5-900a-4c5d58e71b9c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:00.710098 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f40875-350b-41b5-900a-4c5d58e71b9c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:00.715334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:00.716016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:00.716339 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:00.716619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] PATH_INFO: `/v3/users/91b2d8698d2f438ea57162d67f601e31` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:00.717777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91b2d8698d2f438ea57162d67f601e31)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:00.719702 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:00.775636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91b2d8698d2f438ea57162d67f601e31'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:00.783668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91b2d8698d2f438ea57162d67f601e31'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:00.784330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91b2d8698d2f438ea57162d67f601e31'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:00.784969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91b2d8698d2f438ea57162d67f601e31'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:00.792820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91b2d8698d2f438ea57162d67f601e31'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:00.810922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1315/2617] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:30:00 2022] DELETE /identity/v3/users/91b2d8698d2f438ea57162d67f601e31 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:00.815584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984e1e-dea1-44b0-8d9d-538ee19efe5a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:00.815997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984e1e-dea1-44b0-8d9d-538ee19efe5a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:00.816339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2984e1e-dea1-44b0-8d9d-538ee19efe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:00.837972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a2984e1e-dea1-44b0-8d9d-538ee19efe5a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:00.839849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a2984e1e-dea1-44b0-8d9d-538ee19efe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:00.902971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1303/2618] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:00.913062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:00.914556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:00.974960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c881e7-7e91-42c8-9217-f6b734373052 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:00.978794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f16bd-6bcb-4f9c-b323-169521f74f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:00.979402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f16bd-6bcb-4f9c-b323-169521f74f34 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:00.979675 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f16bd-6bcb-4f9c-b323-169521f74f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:00.980001 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f16bd-6bcb-4f9c-b323-169521f74f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:00.989393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f16bd-6bcb-4f9c-b323-169521f74f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:00.990816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f16bd-6bcb-4f9c-b323-169521f74f34 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:01.001566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1316/2619] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:30:01.276082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2984e1e-dea1-44b0-8d9d-538ee19efe5a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:01.277518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a2984e1e-dea1-44b0-8d9d-538ee19efe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:01.290997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2984e1e-dea1-44b0-8d9d-538ee19efe5a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:01.295808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:01.296600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:01.296885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:01.297157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] PATH_INFO: `/v3/projects/05faadd85ad84c629070f527ba67a8df` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:01.298293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] RBAC: Authorizing `identity:delete_project(project_id=05faadd85ad84c629070f527ba67a8df)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:01.300237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:01.353597 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05faadd85ad84c629070f527ba67a8df', 'request_id': 'req-42f83260-ef8e-470e-92de-252000ca788e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:01.361473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05faadd85ad84c629070f527ba67a8df', 'request_id': 'req-42f83260-ef8e-470e-92de-252000ca788e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:01.368668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05faadd85ad84c629070f527ba67a8df', 'request_id': 'req-42f83260-ef8e-470e-92de-252000ca788e'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:01.379570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] The token cache is being invalidate because project 05faadd85ad84c629070f527ba67a8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:30:01.379892 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:01.380514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:01.381992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1304/2620] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:30:01 2022] DELETE /identity/v3/projects/05faadd85ad84c629070f527ba67a8df => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:09.542015 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4681-8e1e-4e4e-9ed5-08e1d3f075e1 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:09.542513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4681-8e1e-4e4e-9ed5-08e1d3f075e1 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:09.542610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a4681-8e1e-4e4e-9ed5-08e1d3f075e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:09.575669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-848a4681-8e1e-4e4e-9ed5-08e1d3f075e1 None None] MFA Rules not processed for user `376fd55b948f49fda100a7da983f285c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:09.576938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-848a4681-8e1e-4e4e-9ed5-08e1d3f075e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:09.595002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:09.636129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1317/2621] 10.222.0.71 () {58 vars in 967 bytes} [Wed Sep 21 12:30:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:09.648653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:09.650151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:09.847711 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f83260-ef8e-470e-92de-252000ca788e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:09.853589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1d5c6-1d90-470a-8f49-52be93179890 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e0bf75c730c946bc9168dd418e7af637', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:09.854209 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1d5c6-1d90-470a-8f49-52be93179890 service swift] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:09.854488 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1d5c6-1d90-470a-8f49-52be93179890 service swift] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:09.854759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1d5c6-1d90-470a-8f49-52be93179890 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:09.866333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1d5c6-1d90-470a-8f49-52be93179890 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:09.867975 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1d5c6-1d90-470a-8f49-52be93179890 service swift] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:09.876304 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1305/2622] 10.222.0.71 () {60 vars in 1384 bytes} [Wed Sep 21 12:30:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 12:30:19.628931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=15 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:19.873217 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:36.544528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4681-8e1e-4e4e-9ed5-08e1d3f075e1 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:36.545975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-848a4681-8e1e-4e4e-9ed5-08e1d3f075e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:36.604845 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a4681-8e1e-4e4e-9ed5-08e1d3f075e1 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:36.608509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1b983-38c5-4420-bc8d-5be25befc47f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:36.609132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1b983-38c5-4420-bc8d-5be25befc47f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:36.609405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1b983-38c5-4420-bc8d-5be25befc47f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:36.609678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1b983-38c5-4420-bc8d-5be25befc47f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:36.611416 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7ef1b983-38c5-4420-bc8d-5be25befc47f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:36.673580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1b983-38c5-4420-bc8d-5be25befc47f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:36.674934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1b983-38c5-4420-bc8d-5be25befc47f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:36.685362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1318/2623] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:30:39.835987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1d5c6-1d90-470a-8f49-52be93179890 service swift] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:39.837617 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cfd1d5c6-1d90-470a-8f49-52be93179890 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:39.854727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1d5c6-1d90-470a-8f49-52be93179890 service swift] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:39.864391 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:39.865860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:39.866424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:39.866991 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] PATH_INFO: `/v3/users/8790c086dcaa4da5b739bcd02d30b4c0` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:39.894955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8790c086dcaa4da5b739bcd02d30b4c0)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:39.897523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:39.957740 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8790c086dcaa4da5b739bcd02d30b4c0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:39.958774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8790c086dcaa4da5b739bcd02d30b4c0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:39.967673 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8790c086dcaa4da5b739bcd02d30b4c0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:39.968344 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8790c086dcaa4da5b739bcd02d30b4c0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:39.975882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-4b83ff92-d75a-470c-bd7b-ebbd8588d086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8790c086dcaa4da5b739bcd02d30b4c0'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:39.994809 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1306/2624] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:30:39 2022] DELETE /identity/v3/users/8790c086dcaa4da5b739bcd02d30b4c0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:40.171133 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1b983-38c5-4420-bc8d-5be25befc47f service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:40.186711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1b983-38c5-4420-bc8d-5be25befc47f service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:40.244403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:40.245032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.245313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.245589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] PATH_INFO: `/v3/projects/9340222cfe154efe8a6064b197d54499` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.255747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9340222cfe154efe8a6064b197d54499)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:40.257680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:40.304879 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:40.309013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9340222cfe154efe8a6064b197d54499', 'request_id': 'req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:40.315598 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9340222cfe154efe8a6064b197d54499', 'request_id': 'req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:40.323582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9340222cfe154efe8a6064b197d54499', 'request_id': 'req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:40.336440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] The token cache is being invalidate because project 9340222cfe154efe8a6064b197d54499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:30:40.336729 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:40.338028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c72d91e1-88c3-4c7c-8598-cd9bfcf12e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:40.339608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1319/2625] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:30:40 2022] DELETE /identity/v3/projects/9340222cfe154efe8a6064b197d54499 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:40.344636 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada402-adad-4472-9fa7-3f398466745d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.344865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada402-adad-4472-9fa7-3f398466745d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.345142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ada402-adad-4472-9fa7-3f398466745d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.364639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-98ada402-adad-4472-9fa7-3f398466745d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:40.365841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-98ada402-adad-4472-9fa7-3f398466745d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:40.423736 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1307/2626] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:30:40.432644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e7e60-2a7a-4cd7-82e7-a32e9acc4ad0 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.432935 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e7e60-2a7a-4cd7-82e7-a32e9acc4ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.433214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e7e60-2a7a-4cd7-82e7-a32e9acc4ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.453091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3c9e7e60-2a7a-4cd7-82e7-a32e9acc4ad0 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:40.454286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3c9e7e60-2a7a-4cd7-82e7-a32e9acc4ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:40.462040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1320/2627] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:40.466749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada402-adad-4472-9fa7-3f398466745d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:40.476145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ada402-adad-4472-9fa7-3f398466745d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:40.481534 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6faf0-c417-4de6-b7bf-60d963e23135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:40.482138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6faf0-c417-4de6-b7bf-60d963e23135 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.482425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6faf0-c417-4de6-b7bf-60d963e23135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.482695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6faf0-c417-4de6-b7bf-60d963e23135 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.483372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6faf0-c417-4de6-b7bf-60d963e23135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:30:40.483742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6faf0-c417-4de6-b7bf-60d963e23135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:40.485274 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6faf0-c417-4de6-b7bf-60d963e23135 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:40.497032 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1308/2628] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:30:40.501294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e7e60-2a7a-4cd7-82e7-a32e9acc4ad0 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:40.512343 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e7e60-2a7a-4cd7-82e7-a32e9acc4ad0 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:40.517023 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71a860-3f63-47ab-97ab-1489994ce56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:40.517626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71a860-3f63-47ab-97ab-1489994ce56a admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.517909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71a860-3f63-47ab-97ab-1489994ce56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.518181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71a860-3f63-47ab-97ab-1489994ce56a admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.518816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71a860-3f63-47ab-97ab-1489994ce56a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:40.520764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71a860-3f63-47ab-97ab-1489994ce56a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:40.558148 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1321/2629] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:30:40 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:40.562401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6faf0-c417-4de6-b7bf-60d963e23135 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:40.573157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6faf0-c417-4de6-b7bf-60d963e23135 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:40.620694 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:40.644474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e24239-24d3-4deb-b359-faa79a8aef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:40.645092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e24239-24d3-4deb-b359-faa79a8aef00 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.645371 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e24239-24d3-4deb-b359-faa79a8aef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.645643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e24239-24d3-4deb-b359-faa79a8aef00 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.646295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e24239-24d3-4deb-b359-faa79a8aef00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:40.648239 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e24239-24d3-4deb-b359-faa79a8aef00 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:40.670199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1309/2630] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:30:40 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:40.674846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71a860-3f63-47ab-97ab-1489994ce56a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:40.685250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71a860-3f63-47ab-97ab-1489994ce56a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:40.689928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6731c-8fac-4e12-92ae-8b1990127ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:40.690588 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6731c-8fac-4e12-92ae-8b1990127ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.691025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6731c-8fac-4e12-92ae-8b1990127ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.691468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6731c-8fac-4e12-92ae-8b1990127ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.692577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6731c-8fac-4e12-92ae-8b1990127ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:40.695358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6731c-8fac-4e12-92ae-8b1990127ff9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:40.706007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1322/2631] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:30:40.710147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e24239-24d3-4deb-b359-faa79a8aef00 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:40.720521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e24239-24d3-4deb-b359-faa79a8aef00 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:40.725046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72123d81-bb82-4766-b74c-5d2be80d5c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:40.725633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72123d81-bb82-4766-b74c-5d2be80d5c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.725906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72123d81-bb82-4766-b74c-5d2be80d5c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.726169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72123d81-bb82-4766-b74c-5d2be80d5c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.726794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72123d81-bb82-4766-b74c-5d2be80d5c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:40.728492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72123d81-bb82-4766-b74c-5d2be80d5c3b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:40.738247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1310/2632] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:30:40.742390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6731c-8fac-4e12-92ae-8b1990127ff9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:40.753025 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6731c-8fac-4e12-92ae-8b1990127ff9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:40.757779 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b5fd-baef-4446-93d6-bcebb8b9195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:40.758432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b5fd-baef-4446-93d6-bcebb8b9195e admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.758706 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b5fd-baef-4446-93d6-bcebb8b9195e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.758976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b5fd-baef-4446-93d6-bcebb8b9195e admin admin] PATH_INFO: `/v3/projects/db082e88e5244cc3825a6f7234bf75e1/users/a02d842c64fb429180f0948e9e3f168c/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.774540 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268b5fd-baef-4446-93d6-bcebb8b9195e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db082e88e5244cc3825a6f7234bf75e1, user_id=a02d842c64fb429180f0948e9e3f168c, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:40.777239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268b5fd-baef-4446-93d6-bcebb8b9195e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:40.790823 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1323/2633] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:30:40 2022] PUT /identity/v3/projects/db082e88e5244cc3825a6f7234bf75e1/users/a02d842c64fb429180f0948e9e3f168c/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:40.801434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca01049-5eaa-44b5-ac6d-4350111a66e1 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.801726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca01049-5eaa-44b5-ac6d-4350111a66e1 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.802007 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca01049-5eaa-44b5-ac6d-4350111a66e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.831535 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4ca01049-5eaa-44b5-ac6d-4350111a66e1 None None] MFA Rules not processed for user `a02d842c64fb429180f0948e9e3f168c`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:40.832806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4ca01049-5eaa-44b5-ac6d-4350111a66e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:40.897275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1311/2634] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:40.901740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b5fd-baef-4446-93d6-bcebb8b9195e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:40.962983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b5fd-baef-4446-93d6-bcebb8b9195e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:40.968068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29646cb-c4bf-4022-9e69-fadf4a9fbc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:40.968676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29646cb-c4bf-4022-9e69-fadf4a9fbc76 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:40.968951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29646cb-c4bf-4022-9e69-fadf4a9fbc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:40.969219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29646cb-c4bf-4022-9e69-fadf4a9fbc76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:40.969921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29646cb-c4bf-4022-9e69-fadf4a9fbc76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:40.973476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29646cb-c4bf-4022-9e69-fadf4a9fbc76 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:41.011662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1324/2635] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:30:40 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:41.012857 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca01049-5eaa-44b5-ac6d-4350111a66e1 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:41.026035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca01049-5eaa-44b5-ac6d-4350111a66e1 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:41.081352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b2e01-e311-4956-9f9b-fd3d8bd44f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:41.081954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b2e01-e311-4956-9f9b-fd3d8bd44f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:41.082230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b2e01-e311-4956-9f9b-fd3d8bd44f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:41.082496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b2e01-e311-4956-9f9b-fd3d8bd44f1a admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:41.083111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b2e01-e311-4956-9f9b-fd3d8bd44f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:41.085026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b2e01-e311-4956-9f9b-fd3d8bd44f1a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:41.106581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1312/2636] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:30:41 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:41.110618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29646cb-c4bf-4022-9e69-fadf4a9fbc76 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:41.121594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29646cb-c4bf-4022-9e69-fadf4a9fbc76 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:41.126199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5f8d1-9d13-40f4-ab7b-c6f9b6485044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:41.126799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5f8d1-9d13-40f4-ab7b-c6f9b6485044 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:41.127081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5f8d1-9d13-40f4-ab7b-c6f9b6485044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:41.127410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5f8d1-9d13-40f4-ab7b-c6f9b6485044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:41.128111 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5f8d1-9d13-40f4-ab7b-c6f9b6485044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:41.129704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5f8d1-9d13-40f4-ab7b-c6f9b6485044 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:41.140342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1325/2637] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:30:41.144483 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b2e01-e311-4956-9f9b-fd3d8bd44f1a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:41.154749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b2e01-e311-4956-9f9b-fd3d8bd44f1a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:41.159232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38947cb4-f9f3-4dcb-8dab-6f2a184e202a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:41.159916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38947cb4-f9f3-4dcb-8dab-6f2a184e202a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:41.160197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38947cb4-f9f3-4dcb-8dab-6f2a184e202a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:41.160469 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38947cb4-f9f3-4dcb-8dab-6f2a184e202a admin admin] PATH_INFO: `/v3/projects/498a2768969d4e93b48e410bf25c9a3e/users/ccf933cf74c5468887941e35fa77b8c3/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:41.175777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38947cb4-f9f3-4dcb-8dab-6f2a184e202a admin admin] RBAC: Authorizing `identity:create_grant(project_id=498a2768969d4e93b48e410bf25c9a3e, user_id=ccf933cf74c5468887941e35fa77b8c3, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:41.178631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38947cb4-f9f3-4dcb-8dab-6f2a184e202a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:41.191660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1313/2638] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:30:41 2022] PUT /identity/v3/projects/498a2768969d4e93b48e410bf25c9a3e/users/ccf933cf74c5468887941e35fa77b8c3/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:41.195626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5f8d1-9d13-40f4-ab7b-c6f9b6485044 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:41.206269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5f8d1-9d13-40f4-ab7b-c6f9b6485044 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:41.237849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:41.268998 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3ea8d-82d4-4051-ab2a-ed9b0041569b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:41.270054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3ea8d-82d4-4051-ab2a-ed9b0041569b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:41.270496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3ea8d-82d4-4051-ab2a-ed9b0041569b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:41.270799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3ea8d-82d4-4051-ab2a-ed9b0041569b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:41.271450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3ea8d-82d4-4051-ab2a-ed9b0041569b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:41.273122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3ea8d-82d4-4051-ab2a-ed9b0041569b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:41.283915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1326/2639] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:30:41.288118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38947cb4-f9f3-4dcb-8dab-6f2a184e202a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:41.299339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38947cb4-f9f3-4dcb-8dab-6f2a184e202a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:41.304075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede3d18-5f7e-4dad-a38d-5881221b03ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:41.304749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3d18-5f7e-4dad-a38d-5881221b03ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:41.305095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3d18-5f7e-4dad-a38d-5881221b03ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:41.305312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede3d18-5f7e-4dad-a38d-5881221b03ee admin admin] PATH_INFO: `/v3/projects/498a2768969d4e93b48e410bf25c9a3e/users/ccf933cf74c5468887941e35fa77b8c3/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:41.307293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede3d18-5f7e-4dad-a38d-5881221b03ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=498a2768969d4e93b48e410bf25c9a3e, user_id=ccf933cf74c5468887941e35fa77b8c3, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:41.310412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede3d18-5f7e-4dad-a38d-5881221b03ee admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:41.322504 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1314/2640] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:30:41 2022] PUT /identity/v3/projects/498a2768969d4e93b48e410bf25c9a3e/users/ccf933cf74c5468887941e35fa77b8c3/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:41.334263 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef9f1-ebcf-44f8-a994-eaeac59b086d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:41.334595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef9f1-ebcf-44f8-a994-eaeac59b086d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:41.334871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef9f1-ebcf-44f8-a994-eaeac59b086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:41.366909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-eb9ef9f1-ebcf-44f8-a994-eaeac59b086d None None] MFA Rules not processed for user `ccf933cf74c5468887941e35fa77b8c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:41.368278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-eb9ef9f1-ebcf-44f8-a994-eaeac59b086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:41.435039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1327/2641] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:41.446177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede3d18-5f7e-4dad-a38d-5881221b03ee admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:41.448955 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7ede3d18-5f7e-4dad-a38d-5881221b03ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:41.461157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:41.509770 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede3d18-5f7e-4dad-a38d-5881221b03ee admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:41.514003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e895d-a211-4b5d-93d5-3ba1ec990e7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:41.514722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e895d-a211-4b5d-93d5-3ba1ec990e7b service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:41.515000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e895d-a211-4b5d-93d5-3ba1ec990e7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:41.515299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e895d-a211-4b5d-93d5-3ba1ec990e7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:41.524730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e895d-a211-4b5d-93d5-3ba1ec990e7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:41.526188 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e895d-a211-4b5d-93d5-3ba1ec990e7b service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:41.535538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1315/2642] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:30:41.612406 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ef9f1-ebcf-44f8-a994-eaeac59b086d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:41.622698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ef9f1-ebcf-44f8-a994-eaeac59b086d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:41.626366 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3ca0e-2088-41b2-aebd-5d00766d361c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:41.627067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3ca0e-2088-41b2-aebd-5d00766d361c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:41.627355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3ca0e-2088-41b2-aebd-5d00766d361c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:41.627627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3ca0e-2088-41b2-aebd-5d00766d361c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:41.685134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3ca0e-2088-41b2-aebd-5d00766d361c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:41.686502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3ca0e-2088-41b2-aebd-5d00766d361c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:41.695377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1328/2643] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:30:51.531665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=22 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:51.601340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e895d-a211-4b5d-93d5-3ba1ec990e7b service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:51.662685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e895d-a211-4b5d-93d5-3ba1ec990e7b service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:51.667404 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:51.668062 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:51.668339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:51.668608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] PATH_INFO: `/v3/users/a02d842c64fb429180f0948e9e3f168c` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:51.669717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02d842c64fb429180f0948e9e3f168c)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:51.671655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:51.691357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=76 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:51.719464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02d842c64fb429180f0948e9e3f168c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:51.720138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02d842c64fb429180f0948e9e3f168c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:51.727249 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02d842c64fb429180f0948e9e3f168c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:51.728454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02d842c64fb429180f0948e9e3f168c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:51.735394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02d842c64fb429180f0948e9e3f168c'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:51.753431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1316/2644] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:30:51 2022] DELETE /identity/v3/users/a02d842c64fb429180f0948e9e3f168c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:51.757967 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f2a46-e544-44b6-ab40-d423fa483941 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:51.758261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f2a46-e544-44b6-ab40-d423fa483941 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:51.758538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f2a46-e544-44b6-ab40-d423fa483941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:51.778437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2d8f2a46-e544-44b6-ab40-d423fa483941 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:51.780003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2d8f2a46-e544-44b6-ab40-d423fa483941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:51.838637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1329/2645] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:51.848615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:51.849998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:51.910006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a54e2-3f90-4855-a03a-f1448ea72c97 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:51.913641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cfd9c-9cd0-44e1-a7ad-fdc279b07e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:51.914242 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cfd9c-9cd0-44e1-a7ad-fdc279b07e6a service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:51.914521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cfd9c-9cd0-44e1-a7ad-fdc279b07e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:51.914793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cfd9c-9cd0-44e1-a7ad-fdc279b07e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:51.924156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0cfd9c-9cd0-44e1-a7ad-fdc279b07e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:51.925569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0cfd9c-9cd0-44e1-a7ad-fdc279b07e6a service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:51.935537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1317/2646] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:30:52.203028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f2a46-e544-44b6-ab40-d423fa483941 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:52.204418 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2d8f2a46-e544-44b6-ab40-d423fa483941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:52.216948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f2a46-e544-44b6-ab40-d423fa483941 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:52.221671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:52.222312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:52.222597 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:52.222880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] PATH_INFO: `/v3/projects/db082e88e5244cc3825a6f7234bf75e1` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:52.224106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db082e88e5244cc3825a6f7234bf75e1)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:52.226029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:52.272671 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db082e88e5244cc3825a6f7234bf75e1', 'request_id': 'req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.279964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db082e88e5244cc3825a6f7234bf75e1', 'request_id': 'req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.288303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db082e88e5244cc3825a6f7234bf75e1', 'request_id': 'req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.299740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] The token cache is being invalidate because project db082e88e5244cc3825a6f7234bf75e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:30:52.300063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.300696 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.302204 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1330/2647] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:30:52 2022] DELETE /identity/v3/projects/db082e88e5244cc3825a6f7234bf75e1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:52.306393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cfd9c-9cd0-44e1-a7ad-fdc279b07e6a service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:52.316517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cfd9c-9cd0-44e1-a7ad-fdc279b07e6a service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:52.335434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:52.375351 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:52.376018 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:52.376301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:52.376576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] PATH_INFO: `/v3/users/ccf933cf74c5468887941e35fa77b8c3` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:52.377692 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf933cf74c5468887941e35fa77b8c3)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:52.379605 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:52.425293 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf933cf74c5468887941e35fa77b8c3'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.425947 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf933cf74c5468887941e35fa77b8c3'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.433174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf933cf74c5468887941e35fa77b8c3'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.433842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf933cf74c5468887941e35fa77b8c3'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.440887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-be0cacf1-c39f-4a24-b54a-2db23c2cfd12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf933cf74c5468887941e35fa77b8c3'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.458345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1318/2648] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:30:52 2022] DELETE /identity/v3/users/ccf933cf74c5468887941e35fa77b8c3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:52.723130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:52.724774 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:52.738306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4263eb5-55b5-4d9b-87ac-028eb3b058a4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:52.796626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:52.797262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:52.797545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:52.797819 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] PATH_INFO: `/v3/projects/498a2768969d4e93b48e410bf25c9a3e` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:52.798933 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=498a2768969d4e93b48e410bf25c9a3e)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:52.800874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:52.833072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:52.851802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '498a2768969d4e93b48e410bf25c9a3e', 'request_id': 'req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.857956 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '498a2768969d4e93b48e410bf25c9a3e', 'request_id': 'req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.865808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '498a2768969d4e93b48e410bf25c9a3e', 'request_id': 'req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.877198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] The token cache is being invalidate because project 498a2768969d4e93b48e410bf25c9a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:30:52.877528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.878280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-783c0f0e-4710-455f-a2f6-ffb1ae9f21f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:52.879800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1331/2649] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:30:52 2022] DELETE /identity/v3/projects/498a2768969d4e93b48e410bf25c9a3e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:52.885306 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459015-b0c8-4890-b63d-1e5c14256f65 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:52.885801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459015-b0c8-4890-b63d-1e5c14256f65 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:52.885924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc459015-b0c8-4890-b63d-1e5c14256f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:52.905326 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-dc459015-b0c8-4890-b63d-1e5c14256f65 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:52.906525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dc459015-b0c8-4890-b63d-1e5c14256f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:52.964089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1319/2650] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:52.972083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d97f4f-a533-40df-a90f-2b96183c43bf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:52.972300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d97f4f-a533-40df-a90f-2b96183c43bf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:52.972576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d97f4f-a533-40df-a90f-2b96183c43bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:52.991887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-a8d97f4f-a533-40df-a90f-2b96183c43bf None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:52.993083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a8d97f4f-a533-40df-a90f-2b96183c43bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:53.001364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1332/2651] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:53.005492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459015-b0c8-4890-b63d-1e5c14256f65 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.015801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc459015-b0c8-4890-b63d-1e5c14256f65 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.020348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31417668-0029-4c17-801a-7bc04b33f59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.020944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31417668-0029-4c17-801a-7bc04b33f59e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.021223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31417668-0029-4c17-801a-7bc04b33f59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.021495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31417668-0029-4c17-801a-7bc04b33f59e admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.022132 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31417668-0029-4c17-801a-7bc04b33f59e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:30:53.022460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31417668-0029-4c17-801a-7bc04b33f59e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.024080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31417668-0029-4c17-801a-7bc04b33f59e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.036000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1320/2652] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:30:53.040502 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d97f4f-a533-40df-a90f-2b96183c43bf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.050307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d97f4f-a533-40df-a90f-2b96183c43bf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.055130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef84318-1062-4ad9-9e2e-e382e1e88b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.056332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef84318-1062-4ad9-9e2e-e382e1e88b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.056640 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef84318-1062-4ad9-9e2e-e382e1e88b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.056919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef84318-1062-4ad9-9e2e-e382e1e88b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.057552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef84318-1062-4ad9-9e2e-e382e1e88b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.059466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef84318-1062-4ad9-9e2e-e382e1e88b39 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.086515 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1333/2653] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:30:53 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:53.090638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31417668-0029-4c17-801a-7bc04b33f59e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.100545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31417668-0029-4c17-801a-7bc04b33f59e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.130052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:53.162844 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76cdf-6e30-41de-bf39-dff7848b7cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.163450 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76cdf-6e30-41de-bf39-dff7848b7cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.163774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76cdf-6e30-41de-bf39-dff7848b7cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.164060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf76cdf-6e30-41de-bf39-dff7848b7cdf admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.164720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf76cdf-6e30-41de-bf39-dff7848b7cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.167130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf76cdf-6e30-41de-bf39-dff7848b7cdf admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.189485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1321/2654] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:30:53 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:53.193791 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef84318-1062-4ad9-9e2e-e382e1e88b39 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.204425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef84318-1062-4ad9-9e2e-e382e1e88b39 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.209034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94225d13-4de0-48e9-88cf-d1e3f7cee8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.209629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94225d13-4de0-48e9-88cf-d1e3f7cee8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.209912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94225d13-4de0-48e9-88cf-d1e3f7cee8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.210188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94225d13-4de0-48e9-88cf-d1e3f7cee8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.210828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94225d13-4de0-48e9-88cf-d1e3f7cee8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.212506 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94225d13-4de0-48e9-88cf-d1e3f7cee8b4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.222634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1334/2655] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:30:53.226452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76cdf-6e30-41de-bf39-dff7848b7cdf admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.237692 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf76cdf-6e30-41de-bf39-dff7848b7cdf admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.242396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f000a-fb7a-4802-a3e8-b4915f5e6592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.242994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f000a-fb7a-4802-a3e8-b4915f5e6592 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.243294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f000a-fb7a-4802-a3e8-b4915f5e6592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.244190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f000a-fb7a-4802-a3e8-b4915f5e6592 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.244829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f000a-fb7a-4802-a3e8-b4915f5e6592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.246414 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f000a-fb7a-4802-a3e8-b4915f5e6592 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.256986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1322/2656] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:30:53.261227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94225d13-4de0-48e9-88cf-d1e3f7cee8b4 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.271405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94225d13-4de0-48e9-88cf-d1e3f7cee8b4 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.276148 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de58bfa-cf5b-4c18-98e9-f8529b4200ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.276857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de58bfa-cf5b-4c18-98e9-f8529b4200ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.277105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de58bfa-cf5b-4c18-98e9-f8529b4200ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.277387 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de58bfa-cf5b-4c18-98e9-f8529b4200ab admin admin] PATH_INFO: `/v3/projects/c9848775e741415282e0f4ff343f9eaf/users/58dc86fc0bf94b09b542c990ab8baf76/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.292373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de58bfa-cf5b-4c18-98e9-f8529b4200ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9848775e741415282e0f4ff343f9eaf, user_id=58dc86fc0bf94b09b542c990ab8baf76, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.295039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de58bfa-cf5b-4c18-98e9-f8529b4200ab admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.309468 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1335/2657] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:30:53 2022] PUT /identity/v3/projects/c9848775e741415282e0f4ff343f9eaf/users/58dc86fc0bf94b09b542c990ab8baf76/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:53.319829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0819d3-34a6-48a2-a211-2e1dd67a596e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.320121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0819d3-34a6-48a2-a211-2e1dd67a596e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.320399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0819d3-34a6-48a2-a211-2e1dd67a596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.349026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-fc0819d3-34a6-48a2-a211-2e1dd67a596e None None] MFA Rules not processed for user `58dc86fc0bf94b09b542c990ab8baf76`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:53.350252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fc0819d3-34a6-48a2-a211-2e1dd67a596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:53.420350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1323/2658] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:53.424968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de58bfa-cf5b-4c18-98e9-f8529b4200ab admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.488770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de58bfa-cf5b-4c18-98e9-f8529b4200ab admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.493284 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c8e17-c357-4ced-b6db-d4fda1040652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.493884 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c8e17-c357-4ced-b6db-d4fda1040652 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.494164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c8e17-c357-4ced-b6db-d4fda1040652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.494437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c8e17-c357-4ced-b6db-d4fda1040652 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.495057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c8e17-c357-4ced-b6db-d4fda1040652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.496987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c8e17-c357-4ced-b6db-d4fda1040652 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.522714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1336/2659] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:30:53 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:53.527307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0819d3-34a6-48a2-a211-2e1dd67a596e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.537837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0819d3-34a6-48a2-a211-2e1dd67a596e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.594706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6a0bc-8756-4f38-9256-5d8126602066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.595313 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6a0bc-8756-4f38-9256-5d8126602066 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.595591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6a0bc-8756-4f38-9256-5d8126602066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.595896 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6a0bc-8756-4f38-9256-5d8126602066 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.596558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6a0bc-8756-4f38-9256-5d8126602066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.598452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6a0bc-8756-4f38-9256-5d8126602066 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.619979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1324/2660] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:30:53 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:30:53.624026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c8e17-c357-4ced-b6db-d4fda1040652 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.634444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c8e17-c357-4ced-b6db-d4fda1040652 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.639080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f3a3f-abb1-4ad1-ab21-f7b6fd3b27f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.639687 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f3a3f-abb1-4ad1-ab21-f7b6fd3b27f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.640012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f3a3f-abb1-4ad1-ab21-f7b6fd3b27f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.640360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f3a3f-abb1-4ad1-ab21-f7b6fd3b27f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.640999 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f3a3f-abb1-4ad1-ab21-f7b6fd3b27f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.642557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f3a3f-abb1-4ad1-ab21-f7b6fd3b27f7 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.653549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1337/2661] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:30:53.657602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6a0bc-8756-4f38-9256-5d8126602066 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.668034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6a0bc-8756-4f38-9256-5d8126602066 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.672641 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9b35-027a-4ac5-8dd1-4ee13cb1f050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.673284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9b35-027a-4ac5-8dd1-4ee13cb1f050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.673561 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9b35-027a-4ac5-8dd1-4ee13cb1f050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.673832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9b35-027a-4ac5-8dd1-4ee13cb1f050 admin admin] PATH_INFO: `/v3/projects/4e8a9cdbf6034fb7b1cee1082c11a868/users/218f1a27164745b0a0f8dc60c31245a9/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.688299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f9b35-027a-4ac5-8dd1-4ee13cb1f050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8a9cdbf6034fb7b1cee1082c11a868, user_id=218f1a27164745b0a0f8dc60c31245a9, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.690927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f9b35-027a-4ac5-8dd1-4ee13cb1f050 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.703421 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1325/2662] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:30:53 2022] PUT /identity/v3/projects/4e8a9cdbf6034fb7b1cee1082c11a868/users/218f1a27164745b0a0f8dc60c31245a9/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:53.707985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f3a3f-abb1-4ad1-ab21-f7b6fd3b27f7 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.718614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f3a3f-abb1-4ad1-ab21-f7b6fd3b27f7 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.738534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:53.776520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee9104-d9b6-48c0-9093-48c1ac6a3dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.777135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee9104-d9b6-48c0-9093-48c1ac6a3dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.777417 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee9104-d9b6-48c0-9093-48c1ac6a3dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.777688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee9104-d9b6-48c0-9093-48c1ac6a3dd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.778322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee9104-d9b6-48c0-9093-48c1ac6a3dd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.779951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee9104-d9b6-48c0-9093-48c1ac6a3dd6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.790475 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1338/2663] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:30:53.794643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9b35-027a-4ac5-8dd1-4ee13cb1f050 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.800111 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:53.806402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9b35-027a-4ac5-8dd1-4ee13cb1f050 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:53.811470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0de66-826c-49a7-a6d7-289665c940bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:53.812145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0de66-826c-49a7-a6d7-289665c940bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.812424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0de66-826c-49a7-a6d7-289665c940bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.812998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0de66-826c-49a7-a6d7-289665c940bb admin admin] PATH_INFO: `/v3/projects/4e8a9cdbf6034fb7b1cee1082c11a868/users/218f1a27164745b0a0f8dc60c31245a9/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.814608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0de66-826c-49a7-a6d7-289665c940bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8a9cdbf6034fb7b1cee1082c11a868, user_id=218f1a27164745b0a0f8dc60c31245a9, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:53.817246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0de66-826c-49a7-a6d7-289665c940bb admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:53.828025 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1326/2664] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:30:53 2022] PUT /identity/v3/projects/4e8a9cdbf6034fb7b1cee1082c11a868/users/218f1a27164745b0a0f8dc60c31245a9/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:53.839631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd06e9-4afb-41e0-a66f-db620981f3a4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:53.839976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd06e9-4afb-41e0-a66f-db620981f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:53.840256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd06e9-4afb-41e0-a66f-db620981f3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:53.869865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c8dd06e9-4afb-41e0-a66f-db620981f3a4 None None] MFA Rules not processed for user `218f1a27164745b0a0f8dc60c31245a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:53.871081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c8dd06e9-4afb-41e0-a66f-db620981f3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:53.937876 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1339/2665] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:53.950618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0de66-826c-49a7-a6d7-289665c940bb admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:53.952443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bad0de66-826c-49a7-a6d7-289665c940bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:54.008901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0de66-826c-49a7-a6d7-289665c940bb admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:54.013065 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acdd0c-b528-4d68-a990-6f82b3355e10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:54.013601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acdd0c-b528-4d68-a990-6f82b3355e10 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:54.013882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acdd0c-b528-4d68-a990-6f82b3355e10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:54.014155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acdd0c-b528-4d68-a990-6f82b3355e10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:54.023013 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acdd0c-b528-4d68-a990-6f82b3355e10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:54.024455 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acdd0c-b528-4d68-a990-6f82b3355e10 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:54.034549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1327/2666] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:30:54.446526 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd06e9-4afb-41e0-a66f-db620981f3a4 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:54.508568 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd06e9-4afb-41e0-a66f-db620981f3a4 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:54.514116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:54.514756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:54.515034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:54.515305 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] PATH_INFO: `/v3/users/58dc86fc0bf94b09b542c990ab8baf76` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:54.516393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58dc86fc0bf94b09b542c990ab8baf76)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:54.518361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:54.563777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58dc86fc0bf94b09b542c990ab8baf76'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:54.565445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:54.571701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58dc86fc0bf94b09b542c990ab8baf76'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:54.572959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58dc86fc0bf94b09b542c990ab8baf76'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:54.573591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58dc86fc0bf94b09b542c990ab8baf76'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:54.580953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58dc86fc0bf94b09b542c990ab8baf76'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:54.598450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1340/2667] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:30:54 2022] DELETE /identity/v3/users/58dc86fc0bf94b09b542c990ab8baf76 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:54.602948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e735-ca22-4331-a4c9-219577474430 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:54.603235 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e735-ca22-4331-a4c9-219577474430 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:54.603510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e735-ca22-4331-a4c9-219577474430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:54.623028 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-dfb8e735-ca22-4331-a4c9-219577474430 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:54.624257 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dfb8e735-ca22-4331-a4c9-219577474430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:54.681721 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1328/2668] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:54.692209 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:54.693610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:54.751551 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61664ef-0921-4a39-b5f2-c5c64744c634 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:54.755257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b75fc-87c4-474e-aedb-5ab03f4044e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:54.755902 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b75fc-87c4-474e-aedb-5ab03f4044e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:54.756186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b75fc-87c4-474e-aedb-5ab03f4044e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:54.756463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b75fc-87c4-474e-aedb-5ab03f4044e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:54.766759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b75fc-87c4-474e-aedb-5ab03f4044e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:54.768248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b75fc-87c4-474e-aedb-5ab03f4044e7 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:54.777899 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1341/2669] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:30:55.044860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8e735-ca22-4331-a4c9-219577474430 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:55.046372 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dfb8e735-ca22-4331-a4c9-219577474430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:55.058199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8e735-ca22-4331-a4c9-219577474430 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:55.062760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:55.063393 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:55.063669 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:55.063977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] PATH_INFO: `/v3/projects/c9848775e741415282e0f4ff343f9eaf` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:55.065512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9848775e741415282e0f4ff343f9eaf)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:55.067491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:55.122171 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9848775e741415282e0f4ff343f9eaf', 'request_id': 'req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.129003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9848775e741415282e0f4ff343f9eaf', 'request_id': 'req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.137506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9848775e741415282e0f4ff343f9eaf', 'request_id': 'req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.148216 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] The token cache is being invalidate because project c9848775e741415282e0f4ff343f9eaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:30:55.148505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.149133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.150611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1329/2670] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:30:55 2022] DELETE /identity/v3/projects/c9848775e741415282e0f4ff343f9eaf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:55.155057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b75fc-87c4-474e-aedb-5ab03f4044e7 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:55.164737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b75fc-87c4-474e-aedb-5ab03f4044e7 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:55.221060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:55.221778 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:55.222056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:55.222426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] PATH_INFO: `/v3/users/218f1a27164745b0a0f8dc60c31245a9` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:55.223655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218f1a27164745b0a0f8dc60c31245a9)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:55.225619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:55.273283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218f1a27164745b0a0f8dc60c31245a9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.281145 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218f1a27164745b0a0f8dc60c31245a9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.281762 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218f1a27164745b0a0f8dc60c31245a9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.282367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218f1a27164745b0a0f8dc60c31245a9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.289507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-31c94949-0022-4080-b41e-28332c773626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218f1a27164745b0a0f8dc60c31245a9'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.295491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:55.310532 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1342/2671] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:30:55 2022] DELETE /identity/v3/users/218f1a27164745b0a0f8dc60c31245a9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:55.580328 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:55.581939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:55.595426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fb094-2c6e-44eb-ab57-ab0b897553fa admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:55.608428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:55.655730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:55.656381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:55.656664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:55.656934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] PATH_INFO: `/v3/projects/4e8a9cdbf6034fb7b1cee1082c11a868` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:55.658099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8a9cdbf6034fb7b1cee1082c11a868)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:55.660081 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:55.706356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8a9cdbf6034fb7b1cee1082c11a868', 'request_id': 'req-bf8a9418-340b-4b0d-a43a-a196304734e9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.712471 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8a9cdbf6034fb7b1cee1082c11a868', 'request_id': 'req-bf8a9418-340b-4b0d-a43a-a196304734e9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.719835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8a9cdbf6034fb7b1cee1082c11a868', 'request_id': 'req-bf8a9418-340b-4b0d-a43a-a196304734e9'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.730295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] The token cache is being invalidate because project 4e8a9cdbf6034fb7b1cee1082c11a868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:30:55.730583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.731229 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bf8a9418-340b-4b0d-a43a-a196304734e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:30:55.732771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1330/2672] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:30:55 2022] DELETE /identity/v3/projects/4e8a9cdbf6034fb7b1cee1082c11a868 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:55.738162 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cc745-8b42-4a7a-8bb1-7c1c1fcfac06 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:55.738480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cc745-8b42-4a7a-8bb1-7c1c1fcfac06 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:55.738773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cc745-8b42-4a7a-8bb1-7c1c1fcfac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:55.759152 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-663cc745-8b42-4a7a-8bb1-7c1c1fcfac06 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:55.760396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-663cc745-8b42-4a7a-8bb1-7c1c1fcfac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:55.819278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1343/2673] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:30:55.828266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99c08f-7894-473a-abb3-2417a1a7c4a2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:55.828493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99c08f-7894-473a-abb3-2417a1a7c4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:55.828771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e99c08f-7894-473a-abb3-2417a1a7c4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:55.848318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-1e99c08f-7894-473a-abb3-2417a1a7c4a2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:55.849494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1e99c08f-7894-473a-abb3-2417a1a7c4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:55.857602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1331/2674] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:55.862060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cc745-8b42-4a7a-8bb1-7c1c1fcfac06 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:55.872395 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cc745-8b42-4a7a-8bb1-7c1c1fcfac06 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:55.876909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be3f3d-1673-49b5-a95a-7f42ca82598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:55.877632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be3f3d-1673-49b5-a95a-7f42ca82598c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:55.877776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be3f3d-1673-49b5-a95a-7f42ca82598c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:55.878048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be3f3d-1673-49b5-a95a-7f42ca82598c admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:55.878682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be3f3d-1673-49b5-a95a-7f42ca82598c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:30:55.878974 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be3f3d-1673-49b5-a95a-7f42ca82598c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:55.880611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be3f3d-1673-49b5-a95a-7f42ca82598c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:55.893003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1344/2675] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:30:55.897310 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99c08f-7894-473a-abb3-2417a1a7c4a2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:55.906422 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e99c08f-7894-473a-abb3-2417a1a7c4a2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:55.910906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dabbb-0bd3-42a0-bed9-38279f99ab1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:55.911494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dabbb-0bd3-42a0-bed9-38279f99ab1d admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:55.911794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dabbb-0bd3-42a0-bed9-38279f99ab1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:55.912060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dabbb-0bd3-42a0-bed9-38279f99ab1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:55.912682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dabbb-0bd3-42a0-bed9-38279f99ab1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:55.914544 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dabbb-0bd3-42a0-bed9-38279f99ab1d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:55.941532 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1332/2676] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:30:55 2022] POST /identity/v3/projects => generated 393 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:55.945864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be3f3d-1673-49b5-a95a-7f42ca82598c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:55.957073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be3f3d-1673-49b5-a95a-7f42ca82598c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.014258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db7a84-627f-4ddb-acc8-ebcacbd693c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.014865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db7a84-627f-4ddb-acc8-ebcacbd693c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.015149 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db7a84-627f-4ddb-acc8-ebcacbd693c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.015428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db7a84-627f-4ddb-acc8-ebcacbd693c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.016118 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db7a84-627f-4ddb-acc8-ebcacbd693c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.018060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db7a84-627f-4ddb-acc8-ebcacbd693c1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.049795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1345/2677] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:30:55 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:56.054292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dabbb-0bd3-42a0-bed9-38279f99ab1d admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.065815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dabbb-0bd3-42a0-bed9-38279f99ab1d admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.070481 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3e7ab-8d1d-457c-ad98-c9fb1a7df95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.071084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3e7ab-8d1d-457c-ad98-c9fb1a7df95e admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.071364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3e7ab-8d1d-457c-ad98-c9fb1a7df95e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.071639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3e7ab-8d1d-457c-ad98-c9fb1a7df95e admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.072560 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3e7ab-8d1d-457c-ad98-c9fb1a7df95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.074163 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3e7ab-8d1d-457c-ad98-c9fb1a7df95e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.084402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1333/2678] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:30:56.088479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db7a84-627f-4ddb-acc8-ebcacbd693c1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.098901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db7a84-627f-4ddb-acc8-ebcacbd693c1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.103567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b76bd2-3989-4950-a947-b5df574d797b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.104689 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b76bd2-3989-4950-a947-b5df574d797b admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.104976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b76bd2-3989-4950-a947-b5df574d797b admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.105261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b76bd2-3989-4950-a947-b5df574d797b admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.105887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b76bd2-3989-4950-a947-b5df574d797b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.107482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b76bd2-3989-4950-a947-b5df574d797b admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.118095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1346/2679] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:30:56.122837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3e7ab-8d1d-457c-ad98-c9fb1a7df95e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.132802 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3e7ab-8d1d-457c-ad98-c9fb1a7df95e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.137517 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48b63e-a918-4cdb-bbab-524def658818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.138187 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48b63e-a918-4cdb-bbab-524def658818 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.138464 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48b63e-a918-4cdb-bbab-524def658818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.138733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48b63e-a918-4cdb-bbab-524def658818 admin admin] PATH_INFO: `/v3/projects/e3a04c692f8848f2875f631673b57142/users/36b8f07947844f718ea0759258cc72a8/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.153722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48b63e-a918-4cdb-bbab-524def658818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a04c692f8848f2875f631673b57142, user_id=36b8f07947844f718ea0759258cc72a8, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.156389 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48b63e-a918-4cdb-bbab-524def658818 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.169584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1334/2680] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:30:56 2022] PUT /identity/v3/projects/e3a04c692f8848f2875f631673b57142/users/36b8f07947844f718ea0759258cc72a8/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:56.180233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36909587-49f0-4356-adad-da71b41f57b2 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.180626 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36909587-49f0-4356-adad-da71b41f57b2 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.181014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36909587-49f0-4356-adad-da71b41f57b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.209673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-36909587-49f0-4356-adad-da71b41f57b2 None None] MFA Rules not processed for user `36b8f07947844f718ea0759258cc72a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:56.210898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-36909587-49f0-4356-adad-da71b41f57b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:56.252193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:56.276325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1347/2681] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:56.280915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48b63e-a918-4cdb-bbab-524def658818 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.330763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:56.351314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48b63e-a918-4cdb-bbab-524def658818 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.356466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98c5ac-f125-4b69-af56-1c5dd1535683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.357059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c5ac-f125-4b69-af56-1c5dd1535683 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.357335 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c5ac-f125-4b69-af56-1c5dd1535683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.357632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98c5ac-f125-4b69-af56-1c5dd1535683 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.358238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98c5ac-f125-4b69-af56-1c5dd1535683 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.360156 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98c5ac-f125-4b69-af56-1c5dd1535683 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.398751 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1335/2682] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:30:56 2022] POST /identity/v3/projects => generated 391 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:56.403386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36909587-49f0-4356-adad-da71b41f57b2 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.414541 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36909587-49f0-4356-adad-da71b41f57b2 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.478457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6f69c-89c8-43d2-b070-89efb2848678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.479071 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6f69c-89c8-43d2-b070-89efb2848678 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.479979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6f69c-89c8-43d2-b070-89efb2848678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.480260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6f69c-89c8-43d2-b070-89efb2848678 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.480890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6f69c-89c8-43d2-b070-89efb2848678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.483028 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6f69c-89c8-43d2-b070-89efb2848678 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.509947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1348/2683] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:30:56 2022] POST /identity/v3/users => generated 302 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:30:56.514026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98c5ac-f125-4b69-af56-1c5dd1535683 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.524161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98c5ac-f125-4b69-af56-1c5dd1535683 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.529305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed7165-8326-4803-9385-8f419c90a5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.529898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed7165-8326-4803-9385-8f419c90a5ee admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.530175 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed7165-8326-4803-9385-8f419c90a5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.530446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed7165-8326-4803-9385-8f419c90a5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.531077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed7165-8326-4803-9385-8f419c90a5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.532675 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed7165-8326-4803-9385-8f419c90a5ee admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.544343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1336/2684] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:30:56.548746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6f69c-89c8-43d2-b070-89efb2848678 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.558894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6f69c-89c8-43d2-b070-89efb2848678 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.563558 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1a2c3-119b-4dc6-b9aa-2d247de7599f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.564248 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1a2c3-119b-4dc6-b9aa-2d247de7599f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.564525 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1a2c3-119b-4dc6-b9aa-2d247de7599f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.564800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1a2c3-119b-4dc6-b9aa-2d247de7599f admin admin] PATH_INFO: `/v3/projects/84a7d32eaf32431782fccdb4962e956b/users/bfee0164ce054890b9c32348287f71b0/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.580534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1a2c3-119b-4dc6-b9aa-2d247de7599f admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a7d32eaf32431782fccdb4962e956b, user_id=bfee0164ce054890b9c32348287f71b0, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.583222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1a2c3-119b-4dc6-b9aa-2d247de7599f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.597767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1349/2685] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:30:56 2022] PUT /identity/v3/projects/84a7d32eaf32431782fccdb4962e956b/users/bfee0164ce054890b9c32348287f71b0/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:56.601936 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed7165-8326-4803-9385-8f419c90a5ee admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.612382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed7165-8326-4803-9385-8f419c90a5ee admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.667819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7cd48-93d0-4d37-b20f-e987cd240988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.668426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7cd48-93d0-4d37-b20f-e987cd240988 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.668703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7cd48-93d0-4d37-b20f-e987cd240988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.668971 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7cd48-93d0-4d37-b20f-e987cd240988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.669599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7cd48-93d0-4d37-b20f-e987cd240988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.671218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7cd48-93d0-4d37-b20f-e987cd240988 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.680806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1337/2686] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:30:56.685114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1a2c3-119b-4dc6-b9aa-2d247de7599f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.695310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1a2c3-119b-4dc6-b9aa-2d247de7599f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.700207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593867f-905e-49b6-8902-05d906d07f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.700857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593867f-905e-49b6-8902-05d906d07f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.701134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593867f-905e-49b6-8902-05d906d07f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.701409 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593867f-905e-49b6-8902-05d906d07f38 admin admin] PATH_INFO: `/v3/projects/84a7d32eaf32431782fccdb4962e956b/users/bfee0164ce054890b9c32348287f71b0/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.703381 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593867f-905e-49b6-8902-05d906d07f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a7d32eaf32431782fccdb4962e956b, user_id=bfee0164ce054890b9c32348287f71b0, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.706048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593867f-905e-49b6-8902-05d906d07f38 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.722437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1350/2687] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:30:56 2022] PUT /identity/v3/projects/84a7d32eaf32431782fccdb4962e956b/users/bfee0164ce054890b9c32348287f71b0/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:30:56.737833 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bacf15-8f62-4992-b102-5145bbe2d135 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.738114 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bacf15-8f62-4992-b102-5145bbe2d135 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.738399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bacf15-8f62-4992-b102-5145bbe2d135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.766445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a1bacf15-8f62-4992-b102-5145bbe2d135 None None] MFA Rules not processed for user `bfee0164ce054890b9c32348287f71b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:30:56.767654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a1bacf15-8f62-4992-b102-5145bbe2d135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:56.835819 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1338/2688] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:30:56 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:30:56.847378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593867f-905e-49b6-8902-05d906d07f38 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.849088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d593867f-905e-49b6-8902-05d906d07f38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:56.906311 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593867f-905e-49b6-8902-05d906d07f38 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.909828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23dba8-527c-463e-84ce-ed1577378fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.910432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23dba8-527c-463e-84ce-ed1577378fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.910709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23dba8-527c-463e-84ce-ed1577378fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.910978 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23dba8-527c-463e-84ce-ed1577378fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:56.919700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23dba8-527c-463e-84ce-ed1577378fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:56.921144 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d23dba8-527c-463e-84ce-ed1577378fa3 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:56.930428 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1351/2689] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:30:56 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:30:56.978642 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bacf15-8f62-4992-b102-5145bbe2d135 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:30:56.980142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a1bacf15-8f62-4992-b102-5145bbe2d135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:30:56.992049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bacf15-8f62-4992-b102-5145bbe2d135 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:30:56.995663 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2432c2c-ad76-4690-8316-3477e2e80e96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:30:56.996265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2432c2c-ad76-4690-8316-3477e2e80e96 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:30:56.996542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2432c2c-ad76-4690-8316-3477e2e80e96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:30:56.996812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2432c2c-ad76-4690-8316-3477e2e80e96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:30:57.030776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:30:57.053800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2432c2c-ad76-4690-8316-3477e2e80e96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:30:57.055223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2432c2c-ad76-4690-8316-3477e2e80e96 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:30:57.064130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1339/2690] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:30:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:00.377824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23dba8-527c-463e-84ce-ed1577378fa3 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:00.417950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:00.443594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23dba8-527c-463e-84ce-ed1577378fa3 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:00.448522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:00.449155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:00.449433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:00.449705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] PATH_INFO: `/v3/users/36b8f07947844f718ea0759258cc72a8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:00.450830 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36b8f07947844f718ea0759258cc72a8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:00.452901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:00.499498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36b8f07947844f718ea0759258cc72a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:00.507866 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36b8f07947844f718ea0759258cc72a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:00.508734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36b8f07947844f718ea0759258cc72a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:00.509839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36b8f07947844f718ea0759258cc72a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:00.523073 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36b8f07947844f718ea0759258cc72a8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:00.541460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1352/2691] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:00 2022] DELETE /identity/v3/users/36b8f07947844f718ea0759258cc72a8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:00.545958 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd300c20-e3b1-4e16-90d9-63f0d0632f9e None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:00.546447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd300c20-e3b1-4e16-90d9-63f0d0632f9e None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:00.546546 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd300c20-e3b1-4e16-90d9-63f0d0632f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:00.565232 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-cd300c20-e3b1-4e16-90d9-63f0d0632f9e None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:00.566431 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cd300c20-e3b1-4e16-90d9-63f0d0632f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:00.625614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1340/2692] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:31:00.635407 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:00.637247 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:00.694523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ace92e-99ea-459b-855b-a45a6202cef2 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:00.698433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ee35-adcc-47c1-a34c-d6176811ff1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:00.699037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ee35-adcc-47c1-a34c-d6176811ff1c service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:00.699322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ee35-adcc-47c1-a34c-d6176811ff1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:00.699594 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6ee35-adcc-47c1-a34c-d6176811ff1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:00.709011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6ee35-adcc-47c1-a34c-d6176811ff1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:00.710449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf6ee35-adcc-47c1-a34c-d6176811ff1c service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:00.720496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1353/2693] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:00.994615 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd300c20-e3b1-4e16-90d9-63f0d0632f9e None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:00.996122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-cd300c20-e3b1-4e16-90d9-63f0d0632f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:01.008394 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd300c20-e3b1-4e16-90d9-63f0d0632f9e None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:01.013682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:01.014317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:01.014591 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:01.014856 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] PATH_INFO: `/v3/projects/e3a04c692f8848f2875f631673b57142` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:01.015986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a04c692f8848f2875f631673b57142)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:01.017868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:01.063862 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a04c692f8848f2875f631673b57142', 'request_id': 'req-52c46304-cd0b-46b2-bf72-38b41643ba3f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.070016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a04c692f8848f2875f631673b57142', 'request_id': 'req-52c46304-cd0b-46b2-bf72-38b41643ba3f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.078434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a04c692f8848f2875f631673b57142', 'request_id': 'req-52c46304-cd0b-46b2-bf72-38b41643ba3f'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.089554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] The token cache is being invalidate because project e3a04c692f8848f2875f631673b57142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:01.089837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.090457 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.092005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1341/2694] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:00 2022] DELETE /identity/v3/projects/e3a04c692f8848f2875f631673b57142 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:01.096008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ee35-adcc-47c1-a34c-d6176811ff1c service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:01.105808 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6ee35-adcc-47c1-a34c-d6176811ff1c service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:01.162057 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:01.162688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:01.162966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:01.163245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] PATH_INFO: `/v3/users/bfee0164ce054890b9c32348287f71b0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:01.164363 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfee0164ce054890b9c32348287f71b0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:01.166295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:01.167767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:01.215777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfee0164ce054890b9c32348287f71b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.223195 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfee0164ce054890b9c32348287f71b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.223842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfee0164ce054890b9c32348287f71b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.224449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfee0164ce054890b9c32348287f71b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.231337 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-7877f76e-a02c-482b-9f85-4db258270974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfee0164ce054890b9c32348287f71b0'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.249420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1354/2695] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:01 2022] DELETE /identity/v3/users/bfee0164ce054890b9c32348287f71b0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:01.522859 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:01.524301 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:01.538002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c46304-cd0b-46b2-bf72-38b41643ba3f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:01.598742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:01.599458 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:01.599625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:01.599922 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] PATH_INFO: `/v3/projects/84a7d32eaf32431782fccdb4962e956b` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:01.601029 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a7d32eaf32431782fccdb4962e956b)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:01.602906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:01.604333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:01.650205 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a7d32eaf32431782fccdb4962e956b', 'request_id': 'req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.656905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a7d32eaf32431782fccdb4962e956b', 'request_id': 'req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.664441 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a7d32eaf32431782fccdb4962e956b', 'request_id': 'req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.675352 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] The token cache is being invalidate because project 84a7d32eaf32431782fccdb4962e956b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:01.675639 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.676288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-99aaabdd-3692-4ee8-a4ac-83bddebfcbc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:01.677798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1342/2696] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:01 2022] DELETE /identity/v3/projects/84a7d32eaf32431782fccdb4962e956b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:01.683433 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81f979-7a84-4a04-b259-a92ebfa0f535 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:01.684063 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81f979-7a84-4a04-b259-a92ebfa0f535 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:01.684200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81f979-7a84-4a04-b259-a92ebfa0f535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:01.826355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-9a81f979-7a84-4a04-b259-a92ebfa0f535 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:01.827591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9a81f979-7a84-4a04-b259-a92ebfa0f535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:01.886886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1355/2697] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:01.894727 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1e195-b364-47f4-963c-ed3d1ea6e507 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:01.895015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1e195-b364-47f4-963c-ed3d1ea6e507 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:01.895290 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1e195-b364-47f4-963c-ed3d1ea6e507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:01.916049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4be1e195-b364-47f4-963c-ed3d1ea6e507 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:01.917244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4be1e195-b364-47f4-963c-ed3d1ea6e507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:01.925573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1343/2698] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:01.929853 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81f979-7a84-4a04-b259-a92ebfa0f535 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:01.939648 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81f979-7a84-4a04-b259-a92ebfa0f535 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:01.944372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722d915-bc66-4c7c-856c-79fee87413ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:01.944959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722d915-bc66-4c7c-856c-79fee87413ff admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:01.945234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722d915-bc66-4c7c-856c-79fee87413ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:01.945500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722d915-bc66-4c7c-856c-79fee87413ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:01.946133 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722d915-bc66-4c7c-856c-79fee87413ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:31:01.946430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722d915-bc66-4c7c-856c-79fee87413ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:01.948326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722d915-bc66-4c7c-856c-79fee87413ff admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:01.959650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1356/2699] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:31:01.963835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1e195-b364-47f4-963c-ed3d1ea6e507 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:01.973230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1e195-b364-47f4-963c-ed3d1ea6e507 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:01.977875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d12720-1dcf-4b18-a393-dbcb531699b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:01.978456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d12720-1dcf-4b18-a393-dbcb531699b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:01.978730 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d12720-1dcf-4b18-a393-dbcb531699b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:01.978999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d12720-1dcf-4b18-a393-dbcb531699b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:01.979625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d12720-1dcf-4b18-a393-dbcb531699b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:01.982196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d12720-1dcf-4b18-a393-dbcb531699b5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.010742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1344/2700] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:01 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:31:02.015365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722d915-bc66-4c7c-856c-79fee87413ff admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.025203 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722d915-bc66-4c7c-856c-79fee87413ff admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.081435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943f9da-3877-43ca-b934-7501267813ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.081966 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943f9da-3877-43ca-b934-7501267813ed admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.082249 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943f9da-3877-43ca-b934-7501267813ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.082523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943f9da-3877-43ca-b934-7501267813ed admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.083166 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943f9da-3877-43ca-b934-7501267813ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.085125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943f9da-3877-43ca-b934-7501267813ed admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.097824 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:02.108520 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1357/2701] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:02 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:02.112763 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d12720-1dcf-4b18-a393-dbcb531699b5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.123665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d12720-1dcf-4b18-a393-dbcb531699b5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.128254 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46070ecf-bb1f-48dc-b350-2f26a9ec3683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.128840 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46070ecf-bb1f-48dc-b350-2f26a9ec3683 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.129115 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46070ecf-bb1f-48dc-b350-2f26a9ec3683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.129378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46070ecf-bb1f-48dc-b350-2f26a9ec3683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.130000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46070ecf-bb1f-48dc-b350-2f26a9ec3683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.131562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46070ecf-bb1f-48dc-b350-2f26a9ec3683 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.142174 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1345/2702] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:02.146078 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943f9da-3877-43ca-b934-7501267813ed admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.156584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943f9da-3877-43ca-b934-7501267813ed admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.161085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355fbf2-c813-4a05-a930-e1ab38aaed40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.161677 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355fbf2-c813-4a05-a930-e1ab38aaed40 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.161981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355fbf2-c813-4a05-a930-e1ab38aaed40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.162224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355fbf2-c813-4a05-a930-e1ab38aaed40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.162850 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355fbf2-c813-4a05-a930-e1ab38aaed40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.164464 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355fbf2-c813-4a05-a930-e1ab38aaed40 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.174957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1358/2703] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:02.179172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46070ecf-bb1f-48dc-b350-2f26a9ec3683 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.189118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46070ecf-bb1f-48dc-b350-2f26a9ec3683 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.194068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5caf1-991e-49a8-b6bb-2f83a4567e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.194768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5caf1-991e-49a8-b6bb-2f83a4567e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.195042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5caf1-991e-49a8-b6bb-2f83a4567e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.195314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5caf1-991e-49a8-b6bb-2f83a4567e9a admin admin] PATH_INFO: `/v3/projects/e7468289626d463f9b99c0ba6fe09d80/users/58c322eacd514732a7f6302cf5f94fc8/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.210478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5caf1-991e-49a8-b6bb-2f83a4567e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7468289626d463f9b99c0ba6fe09d80, user_id=58c322eacd514732a7f6302cf5f94fc8, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.213127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5caf1-991e-49a8-b6bb-2f83a4567e9a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.226145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1346/2704] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:02 2022] PUT /identity/v3/projects/e7468289626d463f9b99c0ba6fe09d80/users/58c322eacd514732a7f6302cf5f94fc8/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:02.237737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d09f5-ebad-4f54-88db-2a676c553daf None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.238050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d09f5-ebad-4f54-88db-2a676c553daf None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.238336 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d09f5-ebad-4f54-88db-2a676c553daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.265751 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f95d09f5-ebad-4f54-88db-2a676c553daf None None] MFA Rules not processed for user `58c322eacd514732a7f6302cf5f94fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:02.266960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f95d09f5-ebad-4f54-88db-2a676c553daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:02.332745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1359/2705] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:02.337155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5caf1-991e-49a8-b6bb-2f83a4567e9a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.399098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5caf1-991e-49a8-b6bb-2f83a4567e9a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.403780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeafc0-4faa-4681-97ab-d76c947e7e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.404403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeafc0-4faa-4681-97ab-d76c947e7e79 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.404681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeafc0-4faa-4681-97ab-d76c947e7e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.404954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abeafc0-4faa-4681-97ab-d76c947e7e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.405577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abeafc0-4faa-4681-97ab-d76c947e7e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.407449 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abeafc0-4faa-4681-97ab-d76c947e7e79 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.433347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1347/2706] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:02 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:31:02.438034 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d09f5-ebad-4f54-88db-2a676c553daf None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.449009 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d09f5-ebad-4f54-88db-2a676c553daf None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.509333 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a80100-99df-46c9-9149-e14b17a50222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.510361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a80100-99df-46c9-9149-e14b17a50222 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.510664 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a80100-99df-46c9-9149-e14b17a50222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.510944 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a80100-99df-46c9-9149-e14b17a50222 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.511581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a80100-99df-46c9-9149-e14b17a50222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.514186 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a80100-99df-46c9-9149-e14b17a50222 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.535777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1360/2707] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:02 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:02.539817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeafc0-4faa-4681-97ab-d76c947e7e79 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.549952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abeafc0-4faa-4681-97ab-d76c947e7e79 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.554448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9417db-5f29-4e3d-8645-40dfe8e63a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.555021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9417db-5f29-4e3d-8645-40dfe8e63a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.555295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9417db-5f29-4e3d-8645-40dfe8e63a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.555565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9417db-5f29-4e3d-8645-40dfe8e63a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.556230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9417db-5f29-4e3d-8645-40dfe8e63a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.557820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9417db-5f29-4e3d-8645-40dfe8e63a1a admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.563835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:02.570725 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1348/2708] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:02.574120 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a80100-99df-46c9-9149-e14b17a50222 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.584758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a80100-99df-46c9-9149-e14b17a50222 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.589522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e91a5-f304-4589-9acd-33f882652ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.590471 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e91a5-f304-4589-9acd-33f882652ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.590694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e91a5-f304-4589-9acd-33f882652ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.591007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e91a5-f304-4589-9acd-33f882652ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.591668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e91a5-f304-4589-9acd-33f882652ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.593299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e91a5-f304-4589-9acd-33f882652ed2 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.603587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1361/2709] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:02.607878 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9417db-5f29-4e3d-8645-40dfe8e63a1a admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.617646 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9417db-5f29-4e3d-8645-40dfe8e63a1a admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.622668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff527e0f-d116-4785-b7d6-d7a720420f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.623302 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff527e0f-d116-4785-b7d6-d7a720420f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.623577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff527e0f-d116-4785-b7d6-d7a720420f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.623906 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff527e0f-d116-4785-b7d6-d7a720420f81 admin admin] PATH_INFO: `/v3/projects/a238f598af514f31abb677aaa874923f/users/def3076fccb040008ccf48116692b597/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.638549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff527e0f-d116-4785-b7d6-d7a720420f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a238f598af514f31abb677aaa874923f, user_id=def3076fccb040008ccf48116692b597, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.641180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff527e0f-d116-4785-b7d6-d7a720420f81 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.654061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1349/2710] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:02 2022] PUT /identity/v3/projects/a238f598af514f31abb677aaa874923f/users/def3076fccb040008ccf48116692b597/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:02.664136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405801-bc40-474b-9cb3-a9aa43d60676 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.664424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405801-bc40-474b-9cb3-a9aa43d60676 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.664700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405801-bc40-474b-9cb3-a9aa43d60676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.692451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4e405801-bc40-474b-9cb3-a9aa43d60676 None None] MFA Rules not processed for user `def3076fccb040008ccf48116692b597`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:02.693867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4e405801-bc40-474b-9cb3-a9aa43d60676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:02.736467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:02.761573 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1362/2711] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:02.766836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff527e0f-d116-4785-b7d6-d7a720420f81 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.829476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff527e0f-d116-4785-b7d6-d7a720420f81 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.834105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213152-ac30-4e4b-9a40-45c19d887bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.834699 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213152-ac30-4e4b-9a40-45c19d887bca admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.835027 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213152-ac30-4e4b-9a40-45c19d887bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.835238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213152-ac30-4e4b-9a40-45c19d887bca admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.835893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81213152-ac30-4e4b-9a40-45c19d887bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.837767 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81213152-ac30-4e4b-9a40-45c19d887bca admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.864654 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1350/2712] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:02 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:02.868847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405801-bc40-474b-9cb3-a9aa43d60676 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.880318 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405801-bc40-474b-9cb3-a9aa43d60676 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.937699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f373b-6848-4eb3-96c2-981f135f9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.938309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f373b-6848-4eb3-96c2-981f135f9a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.938716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f373b-6848-4eb3-96c2-981f135f9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.938997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f373b-6848-4eb3-96c2-981f135f9a00 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.939633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f373b-6848-4eb3-96c2-981f135f9a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.941577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f373b-6848-4eb3-96c2-981f135f9a00 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.962767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1363/2713] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:02 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:31:02.966832 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213152-ac30-4e4b-9a40-45c19d887bca admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:02.977268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213152-ac30-4e4b-9a40-45c19d887bca admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:02.982386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8488e-b01f-4709-973f-4e32ba42f9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:02.982980 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8488e-b01f-4709-973f-4e32ba42f9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:02.983259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8488e-b01f-4709-973f-4e32ba42f9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:02.983541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8488e-b01f-4709-973f-4e32ba42f9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:02.984237 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8488e-b01f-4709-973f-4e32ba42f9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:02.985821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8488e-b01f-4709-973f-4e32ba42f9b7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:02.996599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1351/2714] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:03.001224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f373b-6848-4eb3-96c2-981f135f9a00 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:03.011288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f373b-6848-4eb3-96c2-981f135f9a00 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:03.016024 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe52e30-526c-4137-b238-36cf39e05c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:03.016673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe52e30-526c-4137-b238-36cf39e05c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:03.016950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe52e30-526c-4137-b238-36cf39e05c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:03.017226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe52e30-526c-4137-b238-36cf39e05c19 admin admin] PATH_INFO: `/v3/projects/ea78cada8edc4482a3ef389d8b76783f/users/e1a892d6795243bd9c227f752757b78b/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:03.031818 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe52e30-526c-4137-b238-36cf39e05c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea78cada8edc4482a3ef389d8b76783f, user_id=e1a892d6795243bd9c227f752757b78b, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:03.034459 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe52e30-526c-4137-b238-36cf39e05c19 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:03.048599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1364/2715] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:03 2022] PUT /identity/v3/projects/ea78cada8edc4482a3ef389d8b76783f/users/e1a892d6795243bd9c227f752757b78b/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:03.052683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8488e-b01f-4709-973f-4e32ba42f9b7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:03.063676 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8488e-b01f-4709-973f-4e32ba42f9b7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:03.120177 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908d46b-5358-41f8-a455-226e929b1dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:03.120704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908d46b-5358-41f8-a455-226e929b1dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:03.120986 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908d46b-5358-41f8-a455-226e929b1dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:03.121263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908d46b-5358-41f8-a455-226e929b1dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:03.121893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908d46b-5358-41f8-a455-226e929b1dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:03.124057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908d46b-5358-41f8-a455-226e929b1dcd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:03.133651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1352/2716] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:03.138007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe52e30-526c-4137-b238-36cf39e05c19 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:03.148380 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe52e30-526c-4137-b238-36cf39e05c19 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:03.153133 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0902101-51f8-4f33-a181-26694f155a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:03.154359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902101-51f8-4f33-a181-26694f155a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:03.154667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902101-51f8-4f33-a181-26694f155a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:03.154941 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0902101-51f8-4f33-a181-26694f155a1c admin admin] PATH_INFO: `/v3/projects/ea78cada8edc4482a3ef389d8b76783f/users/e1a892d6795243bd9c227f752757b78b/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:03.156936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0902101-51f8-4f33-a181-26694f155a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea78cada8edc4482a3ef389d8b76783f, user_id=e1a892d6795243bd9c227f752757b78b, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:03.159569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0902101-51f8-4f33-a181-26694f155a1c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:03.171680 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1365/2717] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:03 2022] PUT /identity/v3/projects/ea78cada8edc4482a3ef389d8b76783f/users/e1a892d6795243bd9c227f752757b78b/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:03.181992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224ce15-ff1a-46dc-bb4e-31dde6c5db4a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:03.182275 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224ce15-ff1a-46dc-bb4e-31dde6c5db4a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:03.182541 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224ce15-ff1a-46dc-bb4e-31dde6c5db4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:03.210658 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-a224ce15-ff1a-46dc-bb4e-31dde6c5db4a None None] MFA Rules not processed for user `e1a892d6795243bd9c227f752757b78b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:03.211913 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a224ce15-ff1a-46dc-bb4e-31dde6c5db4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:03.245474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:03.281577 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1353/2718] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:03.292952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0902101-51f8-4f33-a181-26694f155a1c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:03.295378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e0902101-51f8-4f33-a181-26694f155a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:03.353741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0902101-51f8-4f33-a181-26694f155a1c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:03.358061 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92e030-9c38-4245-a134-f27a3a94b59c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:03.358724 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92e030-9c38-4245-a134-f27a3a94b59c service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:03.358947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92e030-9c38-4245-a134-f27a3a94b59c service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:03.359217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92e030-9c38-4245-a134-f27a3a94b59c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:03.368162 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92e030-9c38-4245-a134-f27a3a94b59c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:03.369587 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92e030-9c38-4245-a134-f27a3a94b59c service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:03.380288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1366/2719] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:03.426412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224ce15-ff1a-46dc-bb4e-31dde6c5db4a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:03.427845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a224ce15-ff1a-46dc-bb4e-31dde6c5db4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:03.439845 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224ce15-ff1a-46dc-bb4e-31dde6c5db4a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:03.443434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b62547-c246-42a5-ab9c-763ff8d8a685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:03.444060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62547-c246-42a5-ab9c-763ff8d8a685 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:03.444333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62547-c246-42a5-ab9c-763ff8d8a685 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:03.444596 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62547-c246-42a5-ab9c-763ff8d8a685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:03.501094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b62547-c246-42a5-ab9c-763ff8d8a685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:03.502557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b62547-c246-42a5-ab9c-763ff8d8a685 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:03.511551 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1354/2720] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:06.395981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92e030-9c38-4245-a134-f27a3a94b59c service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:06.436938 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:06.459896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92e030-9c38-4245-a134-f27a3a94b59c service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:06.464452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:06.465084 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:06.465364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:06.465639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] PATH_INFO: `/v3/users/58c322eacd514732a7f6302cf5f94fc8` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:06.467834 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c322eacd514732a7f6302cf5f94fc8)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:06.469756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:06.520121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c322eacd514732a7f6302cf5f94fc8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:06.527117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c322eacd514732a7f6302cf5f94fc8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:06.527743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c322eacd514732a7f6302cf5f94fc8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:06.529006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c322eacd514732a7f6302cf5f94fc8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:06.536002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c322eacd514732a7f6302cf5f94fc8'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:06.552914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1367/2721] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:06 2022] DELETE /identity/v3/users/58c322eacd514732a7f6302cf5f94fc8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:06.557490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b305b-3d56-46a1-9a40-734f3a416368 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:06.557978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b305b-3d56-46a1-9a40-734f3a416368 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:06.557978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b305b-3d56-46a1-9a40-734f3a416368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:06.578598 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-e38b305b-3d56-46a1-9a40-734f3a416368 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:06.579853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e38b305b-3d56-46a1-9a40-734f3a416368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:06.639060 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1355/2722] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:06.649022 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:06.650683 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:06.706768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755c0f7-10d0-4104-b512-68698b992ae6 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:06.710354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ecbcc-a99a-4207-84f4-b93b821749db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:06.710950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ecbcc-a99a-4207-84f4-b93b821749db service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:06.711222 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ecbcc-a99a-4207-84f4-b93b821749db service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:06.711488 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ecbcc-a99a-4207-84f4-b93b821749db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:06.721741 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ecbcc-a99a-4207-84f4-b93b821749db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:06.723154 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ecbcc-a99a-4207-84f4-b93b821749db service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:06.723726 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b305b-3d56-46a1-9a40-734f3a416368 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:06.725131 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e38b305b-3d56-46a1-9a40-734f3a416368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:06.733766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1368/2723] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:06.793829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b305b-3d56-46a1-9a40-734f3a416368 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:06.798633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b71a0f-584b-427a-aca0-a1392804f222 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:06.799280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b71a0f-584b-427a-aca0-a1392804f222 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:06.799559 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b71a0f-584b-427a-aca0-a1392804f222 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:06.799852 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b71a0f-584b-427a-aca0-a1392804f222 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:06.801732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-77b71a0f-584b-427a-aca0-a1392804f222 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:06.831835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:06.892533 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b71a0f-584b-427a-aca0-a1392804f222 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:06.893995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b71a0f-584b-427a-aca0-a1392804f222 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:06.913889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1356/2724] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:31:07.011633 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ecbcc-a99a-4207-84f4-b93b821749db service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:07.013141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-565ecbcc-a99a-4207-84f4-b93b821749db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:07.026080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ecbcc-a99a-4207-84f4-b93b821749db service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:07.031325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:07.031991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:07.032289 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:07.032562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] PATH_INFO: `/v3/projects/e7468289626d463f9b99c0ba6fe09d80` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:07.033710 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7468289626d463f9b99c0ba6fe09d80)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:07.035638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:07.085335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7468289626d463f9b99c0ba6fe09d80', 'request_id': 'req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.092198 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7468289626d463f9b99c0ba6fe09d80', 'request_id': 'req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.093948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:07.102083 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7468289626d463f9b99c0ba6fe09d80', 'request_id': 'req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.114565 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] The token cache is being invalidate because project e7468289626d463f9b99c0ba6fe09d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:07.114862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.115503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.117013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1369/2725] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:07 2022] DELETE /identity/v3/projects/e7468289626d463f9b99c0ba6fe09d80 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:31:07.121075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b71a0f-584b-427a-aca0-a1392804f222 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:07.132037 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b71a0f-584b-427a-aca0-a1392804f222 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:07.194005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:07.194973 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:07.195258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:07.195558 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] PATH_INFO: `/v3/users/def3076fccb040008ccf48116692b597` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:07.196772 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=def3076fccb040008ccf48116692b597)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:07.198937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:07.257183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def3076fccb040008ccf48116692b597'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.257838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def3076fccb040008ccf48116692b597'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.267199 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def3076fccb040008ccf48116692b597'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.267784 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def3076fccb040008ccf48116692b597'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.274972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def3076fccb040008ccf48116692b597'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.280403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:07.281915 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:07.293995 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1357/2726] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:07 2022] DELETE /identity/v3/users/def3076fccb040008ccf48116692b597 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:31:07.348280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87e9d8-7f06-4dc0-b4bb-8ec9f45a7682 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:07.399550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d831dc-5a3d-473a-8a42-15d9c48718d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:07.400379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d831dc-5a3d-473a-8a42-15d9c48718d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:07.400658 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d831dc-5a3d-473a-8a42-15d9c48718d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:07.400927 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d831dc-5a3d-473a-8a42-15d9c48718d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:07.402694 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-77d831dc-5a3d-473a-8a42-15d9c48718d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:07.498090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d831dc-5a3d-473a-8a42-15d9c48718d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:07.499549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d831dc-5a3d-473a-8a42-15d9c48718d0 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:07.521716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1370/2727] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:07.617939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:07.619437 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:07.642779 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:07.698277 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1427ec78-0d50-41a9-b2f9-cabf7378d3ef admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:07.703908 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:07.704553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:07.704835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:07.705112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] PATH_INFO: `/v3/projects/a238f598af514f31abb677aaa874923f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:07.706391 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a238f598af514f31abb677aaa874923f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:07.708432 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:07.754607 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a238f598af514f31abb677aaa874923f', 'request_id': 'req-81b6bacc-7344-4381-9277-cfa417a5e931'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.760851 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a238f598af514f31abb677aaa874923f', 'request_id': 'req-81b6bacc-7344-4381-9277-cfa417a5e931'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.769429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a238f598af514f31abb677aaa874923f', 'request_id': 'req-81b6bacc-7344-4381-9277-cfa417a5e931'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.780122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] The token cache is being invalidate because project a238f598af514f31abb677aaa874923f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:07.780409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.781039 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.782518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1358/2728] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:07 2022] DELETE /identity/v3/projects/a238f598af514f31abb677aaa874923f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:07.786487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d831dc-5a3d-473a-8a42-15d9c48718d0 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:07.797126 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d831dc-5a3d-473a-8a42-15d9c48718d0 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:07.817543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:07.859513 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:07.860202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:07.860474 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:07.860917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] PATH_INFO: `/v3/users/e1a892d6795243bd9c227f752757b78b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:07.861936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a892d6795243bd9c227f752757b78b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:07.863910 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:07.911984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a892d6795243bd9c227f752757b78b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.918872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a892d6795243bd9c227f752757b78b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.919507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a892d6795243bd9c227f752757b78b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.920133 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a892d6795243bd9c227f752757b78b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.926962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2dfa65c0-a97a-4140-8729-8fb409781e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a892d6795243bd9c227f752757b78b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:07.944802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1371/2729] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:07 2022] DELETE /identity/v3/users/e1a892d6795243bd9c227f752757b78b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:08.216377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:08.218054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:08.233314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6bacc-7344-4381-9277-cfa417a5e931 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:08.289366 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:08.289982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.290261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.290535 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] PATH_INFO: `/v3/projects/ea78cada8edc4482a3ef389d8b76783f` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.291644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea78cada8edc4482a3ef389d8b76783f)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:08.293618 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:08.340619 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:08.342276 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea78cada8edc4482a3ef389d8b76783f', 'request_id': 'req-274c7bba-0a6f-4eeb-9dec-09f46365d435'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:08.351923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea78cada8edc4482a3ef389d8b76783f', 'request_id': 'req-274c7bba-0a6f-4eeb-9dec-09f46365d435'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:08.360096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea78cada8edc4482a3ef389d8b76783f', 'request_id': 'req-274c7bba-0a6f-4eeb-9dec-09f46365d435'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:08.370540 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] The token cache is being invalidate because project ea78cada8edc4482a3ef389d8b76783f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:08.370821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:08.371448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-274c7bba-0a6f-4eeb-9dec-09f46365d435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:08.373543 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1359/2730] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:08 2022] DELETE /identity/v3/projects/ea78cada8edc4482a3ef389d8b76783f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:08.379652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbefe2-b259-4981-a347-d75a7cf8dbea None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.379975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbefe2-b259-4981-a347-d75a7cf8dbea None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.380254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbefe2-b259-4981-a347-d75a7cf8dbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.401100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3ddbefe2-b259-4981-a347-d75a7cf8dbea None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:08.402243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3ddbefe2-b259-4981-a347-d75a7cf8dbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:08.460869 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1372/2731] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:08.470686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563e8d8-a987-483d-92e3-d314b12e83ea None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.470972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563e8d8-a987-483d-92e3-d314b12e83ea None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.471247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563e8d8-a987-483d-92e3-d314b12e83ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.491287 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9563e8d8-a987-483d-92e3-d314b12e83ea None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:08.492550 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9563e8d8-a987-483d-92e3-d314b12e83ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:08.500312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1360/2732] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:08.504772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbefe2-b259-4981-a347-d75a7cf8dbea None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:08.515527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbefe2-b259-4981-a347-d75a7cf8dbea None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:08.520344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8747e-32eb-46a8-9264-1927c0c35770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:08.520946 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8747e-32eb-46a8-9264-1927c0c35770 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.521224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8747e-32eb-46a8-9264-1927c0c35770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.521500 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8747e-32eb-46a8-9264-1927c0c35770 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.522147 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8747e-32eb-46a8-9264-1927c0c35770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:31:08.522451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8747e-32eb-46a8-9264-1927c0c35770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:08.524081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8747e-32eb-46a8-9264-1927c0c35770 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:08.537104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1373/2733] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:31:08.541631 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563e8d8-a987-483d-92e3-d314b12e83ea None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:08.551339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563e8d8-a987-483d-92e3-d314b12e83ea None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:08.556089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4f53d-590a-4f51-b481-90a771ddb0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:08.556679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f53d-590a-4f51-b481-90a771ddb0cd admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.556953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f53d-590a-4f51-b481-90a771ddb0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.557218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4f53d-590a-4f51-b481-90a771ddb0cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.557843 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4f53d-590a-4f51-b481-90a771ddb0cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:08.559778 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4f53d-590a-4f51-b481-90a771ddb0cd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:08.588655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1361/2734] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:08 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:08.593512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8747e-32eb-46a8-9264-1927c0c35770 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:08.603936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8747e-32eb-46a8-9264-1927c0c35770 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:08.629781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:08.667141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69e217-d026-4e5b-919b-bc549f1733ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:08.667668 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69e217-d026-4e5b-919b-bc549f1733ef admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.668012 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69e217-d026-4e5b-919b-bc549f1733ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.668288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69e217-d026-4e5b-919b-bc549f1733ef admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.668918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69e217-d026-4e5b-919b-bc549f1733ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:08.671539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69e217-d026-4e5b-919b-bc549f1733ef admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:08.697362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1374/2735] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:08 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:08.698949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4f53d-590a-4f51-b481-90a771ddb0cd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:08.709999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4f53d-590a-4f51-b481-90a771ddb0cd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:08.715345 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a38336-8195-4a33-9dc7-d541470b6a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:08.716500 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a38336-8195-4a33-9dc7-d541470b6a3c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.716776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a38336-8195-4a33-9dc7-d541470b6a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.717052 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a38336-8195-4a33-9dc7-d541470b6a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.717701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a38336-8195-4a33-9dc7-d541470b6a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:08.719294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a38336-8195-4a33-9dc7-d541470b6a3c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:08.732808 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1362/2736] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:08.734693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69e217-d026-4e5b-919b-bc549f1733ef admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:08.743744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a38336-8195-4a33-9dc7-d541470b6a3c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:08.745136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69e217-d026-4e5b-919b-bc549f1733ef admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:08.745686 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b1a38336-8195-4a33-9dc7-d541470b6a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:08.750099 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc20fa4-b36a-4027-9ffc-56cf5d7d2add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:08.750705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20fa4-b36a-4027-9ffc-56cf5d7d2add admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.750992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20fa4-b36a-4027-9ffc-56cf5d7d2add admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.751265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc20fa4-b36a-4027-9ffc-56cf5d7d2add admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.751945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc20fa4-b36a-4027-9ffc-56cf5d7d2add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:08.753622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc20fa4-b36a-4027-9ffc-56cf5d7d2add admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:08.765265 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1375/2737] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:08.769430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc20fa4-b36a-4027-9ffc-56cf5d7d2add admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:08.780302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc20fa4-b36a-4027-9ffc-56cf5d7d2add admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:08.784925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c1ad4-77a4-4bf0-b74e-387691a0b5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:08.785570 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c1ad4-77a4-4bf0-b74e-387691a0b5c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.785846 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c1ad4-77a4-4bf0-b74e-387691a0b5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.786117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c1ad4-77a4-4bf0-b74e-387691a0b5c4 admin admin] PATH_INFO: `/v3/projects/7d41e0b5f5154d45bd5583c114ca13c4/users/ba99457e9bc74ee48bda0059472d3755/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.801077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c1ad4-77a4-4bf0-b74e-387691a0b5c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d41e0b5f5154d45bd5583c114ca13c4, user_id=ba99457e9bc74ee48bda0059472d3755, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:08.803963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c1ad4-77a4-4bf0-b74e-387691a0b5c4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:08.814323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a38336-8195-4a33-9dc7-d541470b6a3c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:08.818755 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1376/2738] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:08 2022] PUT /identity/v3/projects/7d41e0b5f5154d45bd5583c114ca13c4/users/ba99457e9bc74ee48bda0059472d3755/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:08.831527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9765-9924-4a1e-aae6-0eb69da199b9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.831527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9765-9924-4a1e-aae6-0eb69da199b9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.832011 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b9765-9924-4a1e-aae6-0eb69da199b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.859456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-5a3b9765-9924-4a1e-aae6-0eb69da199b9 None None] MFA Rules not processed for user `ba99457e9bc74ee48bda0059472d3755`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:08.860799 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5a3b9765-9924-4a1e-aae6-0eb69da199b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:08.876384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a13081-6d03-4f94-ae1a-aadd8d7303aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:08.877003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13081-6d03-4f94-ae1a-aadd8d7303aa service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:08.877285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13081-6d03-4f94-ae1a-aadd8d7303aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:08.877724 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a13081-6d03-4f94-ae1a-aadd8d7303aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:08.879506 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f4a13081-6d03-4f94-ae1a-aadd8d7303aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:08.930969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1377/2739] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:08.942254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9765-9924-4a1e-aae6-0eb69da199b9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:08.944076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5a3b9765-9924-4a1e-aae6-0eb69da199b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:08.949891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:08.954512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a13081-6d03-4f94-ae1a-aadd8d7303aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:08.956151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a13081-6d03-4f94-ae1a-aadd8d7303aa service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:08.969513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1363/2740] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:09.002708 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b9765-9924-4a1e-aae6-0eb69da199b9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:09.006635 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f7cbf-6974-4036-9d08-d5814a2d2032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:09.007245 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f7cbf-6974-4036-9d08-d5814a2d2032 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:09.007524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f7cbf-6974-4036-9d08-d5814a2d2032 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:09.007859 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f7cbf-6974-4036-9d08-d5814a2d2032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:09.018415 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f7cbf-6974-4036-9d08-d5814a2d2032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:09.020360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f7cbf-6974-4036-9d08-d5814a2d2032 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:09.030758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1378/2741] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:31:09.049753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a13081-6d03-4f94-ae1a-aadd8d7303aa service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:09.116350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a13081-6d03-4f94-ae1a-aadd8d7303aa service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:09.121311 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:09.121950 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:09.122760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:09.122999 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] PATH_INFO: `/v3/users/ba99457e9bc74ee48bda0059472d3755` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:09.124192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba99457e9bc74ee48bda0059472d3755)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:09.126104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:09.182509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba99457e9bc74ee48bda0059472d3755'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.183133 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba99457e9bc74ee48bda0059472d3755'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.195001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba99457e9bc74ee48bda0059472d3755'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.195244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba99457e9bc74ee48bda0059472d3755'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.203952 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba99457e9bc74ee48bda0059472d3755'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.221494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1364/2742] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:09 2022] DELETE /identity/v3/users/ba99457e9bc74ee48bda0059472d3755 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:09.226286 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea4c3a-ee80-4da4-8c93-31ed2195903d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:09.226634 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea4c3a-ee80-4da4-8c93-31ed2195903d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:09.227019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea4c3a-ee80-4da4-8c93-31ed2195903d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:09.245660 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-58ea4c3a-ee80-4da4-8c93-31ed2195903d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:09.246864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-58ea4c3a-ee80-4da4-8c93-31ed2195903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:09.257105 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:09.310784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1379/2743] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:09.320314 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:09.321925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:09.383556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10026c-edf9-4020-bd13-23fb21f350f3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:09.387154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434b65f-c041-4155-ba59-18e2f3727a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:09.387771 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434b65f-c041-4155-ba59-18e2f3727a67 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:09.388083 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434b65f-c041-4155-ba59-18e2f3727a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:09.388350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434b65f-c041-4155-ba59-18e2f3727a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:09.397527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434b65f-c041-4155-ba59-18e2f3727a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:09.398939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434b65f-c041-4155-ba59-18e2f3727a67 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:09.409753 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1365/2744] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:09.692300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea4c3a-ee80-4da4-8c93-31ed2195903d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:09.693829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-58ea4c3a-ee80-4da4-8c93-31ed2195903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:09.706875 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea4c3a-ee80-4da4-8c93-31ed2195903d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:09.711569 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:09.712476 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:09.712917 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:09.713344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] PATH_INFO: `/v3/projects/7d41e0b5f5154d45bd5583c114ca13c4` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:09.714693 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d41e0b5f5154d45bd5583c114ca13c4)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:09.718391 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:09.777074 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d41e0b5f5154d45bd5583c114ca13c4', 'request_id': 'req-137058a8-7c34-4271-a1fa-d8ef3d5327de'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.783590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d41e0b5f5154d45bd5583c114ca13c4', 'request_id': 'req-137058a8-7c34-4271-a1fa-d8ef3d5327de'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.791913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d41e0b5f5154d45bd5583c114ca13c4', 'request_id': 'req-137058a8-7c34-4271-a1fa-d8ef3d5327de'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.804610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] The token cache is being invalidate because project 7d41e0b5f5154d45bd5583c114ca13c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:09.804904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.805556 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-137058a8-7c34-4271-a1fa-d8ef3d5327de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:09.810787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1380/2745] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:09 2022] DELETE /identity/v3/projects/7d41e0b5f5154d45bd5583c114ca13c4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:09.812633 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cc02e-be03-4efd-9e27-e7d23a590ace None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:09.813096 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cc02e-be03-4efd-9e27-e7d23a590ace None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:09.813292 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cc02e-be03-4efd-9e27-e7d23a590ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:09.833982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-4a6cc02e-be03-4efd-9e27-e7d23a590ace None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:09.835196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4a6cc02e-be03-4efd-9e27-e7d23a590ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:09.845569 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:09.908127 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1366/2746] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:09.917527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd5f0f-a884-48b9-abb0-dafa538b8a8c None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:09.917896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd5f0f-a884-48b9-abb0-dafa538b8a8c None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:09.918239 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd5f0f-a884-48b9-abb0-dafa538b8a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:09.960619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-82bd5f0f-a884-48b9-abb0-dafa538b8a8c None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:09.964548 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-82bd5f0f-a884-48b9-abb0-dafa538b8a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:09.980120 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1381/2747] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:09.985070 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cc02e-be03-4efd-9e27-e7d23a590ace None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:09.996280 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cc02e-be03-4efd-9e27-e7d23a590ace None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.004501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be637b6c-7fc4-4e05-bb83-78ca0491bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.005173 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be637b6c-7fc4-4e05-bb83-78ca0491bfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.005498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be637b6c-7fc4-4e05-bb83-78ca0491bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.005774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be637b6c-7fc4-4e05-bb83-78ca0491bfd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.007050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be637b6c-7fc4-4e05-bb83-78ca0491bfd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:31:10.007359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be637b6c-7fc4-4e05-bb83-78ca0491bfd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.009666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be637b6c-7fc4-4e05-bb83-78ca0491bfd9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.033220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1367/2748] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 12:31:10.037270 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd5f0f-a884-48b9-abb0-dafa538b8a8c None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.048431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd5f0f-a884-48b9-abb0-dafa538b8a8c None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.055298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b971c10-b956-4694-b50d-d54741d6eafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.055968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b971c10-b956-4694-b50d-d54741d6eafa admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.056384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b971c10-b956-4694-b50d-d54741d6eafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.056659 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b971c10-b956-4694-b50d-d54741d6eafa admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.057521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b971c10-b956-4694-b50d-d54741d6eafa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.059545 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b971c10-b956-4694-b50d-d54741d6eafa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.099549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1382/2749] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:10 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:10.101436 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be637b6c-7fc4-4e05-bb83-78ca0491bfd9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.126447 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be637b6c-7fc4-4e05-bb83-78ca0491bfd9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.221325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ea0f7-9a09-41f9-a974-f8a70e79233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.222233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ea0f7-9a09-41f9-a974-f8a70e79233e admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.222572 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ea0f7-9a09-41f9-a974-f8a70e79233e admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.223109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ea0f7-9a09-41f9-a974-f8a70e79233e admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.223830 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ea0f7-9a09-41f9-a974-f8a70e79233e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.225842 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3ea0f7-9a09-41f9-a974-f8a70e79233e admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.252349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1368/2750] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:10 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:10.256837 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b971c10-b956-4694-b50d-d54741d6eafa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.267472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b971c10-b956-4694-b50d-d54741d6eafa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.272578 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c161a1-9d28-4b9e-9697-60f83f453255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.273394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c161a1-9d28-4b9e-9697-60f83f453255 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.273749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c161a1-9d28-4b9e-9697-60f83f453255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.274104 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c161a1-9d28-4b9e-9697-60f83f453255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.274820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c161a1-9d28-4b9e-9697-60f83f453255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.276531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c161a1-9d28-4b9e-9697-60f83f453255 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.290890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1383/2751] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:10.291664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ea0f7-9a09-41f9-a974-f8a70e79233e admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.305263 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ea0f7-9a09-41f9-a974-f8a70e79233e admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.310691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f42d34-bae5-418d-8ebf-ae6110a83ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.311303 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f42d34-bae5-418d-8ebf-ae6110a83ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.311583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f42d34-bae5-418d-8ebf-ae6110a83ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.312005 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f42d34-bae5-418d-8ebf-ae6110a83ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.312643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f42d34-bae5-418d-8ebf-ae6110a83ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.314226 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f42d34-bae5-418d-8ebf-ae6110a83ce7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.325587 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1369/2752] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:31:10.329688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c161a1-9d28-4b9e-9697-60f83f453255 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.340093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c161a1-9d28-4b9e-9697-60f83f453255 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.344929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54f53a-7add-425d-ac46-584a13fb9c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.345574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54f53a-7add-425d-ac46-584a13fb9c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.345855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54f53a-7add-425d-ac46-584a13fb9c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.346127 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54f53a-7add-425d-ac46-584a13fb9c00 admin admin] PATH_INFO: `/v3/projects/cde8896097b748078b927cf8a6f8aa43/users/a440ccd9eade4b3a9bc671b275b98ff6/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.366130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54f53a-7add-425d-ac46-584a13fb9c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde8896097b748078b927cf8a6f8aa43, user_id=a440ccd9eade4b3a9bc671b275b98ff6, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.368841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54f53a-7add-425d-ac46-584a13fb9c00 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.383322 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1384/2753] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:10 2022] PUT /identity/v3/projects/cde8896097b748078b927cf8a6f8aa43/users/a440ccd9eade4b3a9bc671b275b98ff6/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:10.451510 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd90a38-367f-41cf-bb48-b4cc949d40a6 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.451818 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd90a38-367f-41cf-bb48-b4cc949d40a6 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.452098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd90a38-367f-41cf-bb48-b4cc949d40a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.479682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-0dd90a38-367f-41cf-bb48-b4cc949d40a6 None None] MFA Rules not processed for user `a440ccd9eade4b3a9bc671b275b98ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:10.481082 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-0dd90a38-367f-41cf-bb48-b4cc949d40a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:10.558342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1370/2754] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:10.563691 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54f53a-7add-425d-ac46-584a13fb9c00 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.611455 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:10.649926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54f53a-7add-425d-ac46-584a13fb9c00 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.655300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285882a-3431-44ac-a041-fca3c8a889e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.656054 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285882a-3431-44ac-a041-fca3c8a889e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.656405 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285882a-3431-44ac-a041-fca3c8a889e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.656685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285882a-3431-44ac-a041-fca3c8a889e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.657329 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285882a-3431-44ac-a041-fca3c8a889e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.659238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285882a-3431-44ac-a041-fca3c8a889e8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.687636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1385/2755] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:10 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:31:10.691940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd90a38-367f-41cf-bb48-b4cc949d40a6 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.705075 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd90a38-367f-41cf-bb48-b4cc949d40a6 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.713036 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:10.766339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c3353-8a4e-4b77-b6b1-5673f0cd30e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.766987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c3353-8a4e-4b77-b6b1-5673f0cd30e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.767268 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c3353-8a4e-4b77-b6b1-5673f0cd30e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.767630 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c3353-8a4e-4b77-b6b1-5673f0cd30e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.768339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c3353-8a4e-4b77-b6b1-5673f0cd30e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.770270 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c3353-8a4e-4b77-b6b1-5673f0cd30e4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.794925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1371/2756] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:10 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:10.799096 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285882a-3431-44ac-a041-fca3c8a889e8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.810472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285882a-3431-44ac-a041-fca3c8a889e8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.816293 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88000aa6-7caa-466c-a2af-63291b122126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.816916 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88000aa6-7caa-466c-a2af-63291b122126 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.817233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88000aa6-7caa-466c-a2af-63291b122126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.817497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88000aa6-7caa-466c-a2af-63291b122126 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.818141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88000aa6-7caa-466c-a2af-63291b122126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.819740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88000aa6-7caa-466c-a2af-63291b122126 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.832217 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1386/2757] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:10.836616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c3353-8a4e-4b77-b6b1-5673f0cd30e4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.847112 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c3353-8a4e-4b77-b6b1-5673f0cd30e4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.851632 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c26799-e8d1-437c-ae01-510673bd09a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.852367 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c26799-e8d1-437c-ae01-510673bd09a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.852637 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c26799-e8d1-437c-ae01-510673bd09a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.852905 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c26799-e8d1-437c-ae01-510673bd09a3 admin admin] PATH_INFO: `/v3/projects/0ec3a0f6e94f4f90ae198ace742abbee/users/24ebcb6773c740ee98bdc44f16dbbe51/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.870461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c26799-e8d1-437c-ae01-510673bd09a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec3a0f6e94f4f90ae198ace742abbee, user_id=24ebcb6773c740ee98bdc44f16dbbe51, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.873157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c26799-e8d1-437c-ae01-510673bd09a3 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.887384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1372/2758] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:10 2022] PUT /identity/v3/projects/0ec3a0f6e94f4f90ae198ace742abbee/users/24ebcb6773c740ee98bdc44f16dbbe51/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:10.891457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88000aa6-7caa-466c-a2af-63291b122126 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.902307 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88000aa6-7caa-466c-a2af-63291b122126 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.965796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148771b1-f65e-4728-a97e-e115c458bb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:10.966911 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148771b1-f65e-4728-a97e-e115c458bb47 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:10.967193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148771b1-f65e-4728-a97e-e115c458bb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:10.967466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148771b1-f65e-4728-a97e-e115c458bb47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:10.968153 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148771b1-f65e-4728-a97e-e115c458bb47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:10.969788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148771b1-f65e-4728-a97e-e115c458bb47 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:10.980566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1387/2759] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:31:10.985054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c26799-e8d1-437c-ae01-510673bd09a3 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:10.995307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c26799-e8d1-437c-ae01-510673bd09a3 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:10.999925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb13d8f-b299-4589-878b-a6231a251d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:11.000512 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb13d8f-b299-4589-878b-a6231a251d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:11.000788 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb13d8f-b299-4589-878b-a6231a251d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:11.001057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb13d8f-b299-4589-878b-a6231a251d29 admin admin] PATH_INFO: `/v3/projects/0ec3a0f6e94f4f90ae198ace742abbee/users/24ebcb6773c740ee98bdc44f16dbbe51/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:11.002920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb13d8f-b299-4589-878b-a6231a251d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec3a0f6e94f4f90ae198ace742abbee, user_id=24ebcb6773c740ee98bdc44f16dbbe51, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:11.005542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb13d8f-b299-4589-878b-a6231a251d29 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:11.016599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1373/2760] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:10 2022] PUT /identity/v3/projects/0ec3a0f6e94f4f90ae198ace742abbee/users/24ebcb6773c740ee98bdc44f16dbbe51/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:11.026803 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced83fc9-5a4c-44de-923e-bd4aeabbfb94 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:11.027088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced83fc9-5a4c-44de-923e-bd4aeabbfb94 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:11.027360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced83fc9-5a4c-44de-923e-bd4aeabbfb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:11.055783 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-ced83fc9-5a4c-44de-923e-bd4aeabbfb94 None None] MFA Rules not processed for user `24ebcb6773c740ee98bdc44f16dbbe51`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:11.057065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ced83fc9-5a4c-44de-923e-bd4aeabbfb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:11.130890 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1388/2761] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:11.145104 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb13d8f-b299-4589-878b-a6231a251d29 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:11.146867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4eb13d8f-b299-4589-878b-a6231a251d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:11.204562 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb13d8f-b299-4589-878b-a6231a251d29 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:11.208211 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa6dc0-73fc-40fe-aa3f-3b39d94b1104 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:11.208814 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa6dc0-73fc-40fe-aa3f-3b39d94b1104 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:11.209092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa6dc0-73fc-40fe-aa3f-3b39d94b1104 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:11.209361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa6dc0-73fc-40fe-aa3f-3b39d94b1104 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:11.271348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa6dc0-73fc-40fe-aa3f-3b39d94b1104 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:11.273092 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa6dc0-73fc-40fe-aa3f-3b39d94b1104 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:11.284119 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1374/2762] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:14.059617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced83fc9-5a4c-44de-923e-bd4aeabbfb94 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:14.090420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:14.123997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced83fc9-5a4c-44de-923e-bd4aeabbfb94 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:14.128848 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:14.130397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:14.130397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:14.130397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] PATH_INFO: `/v3/users/a440ccd9eade4b3a9bc671b275b98ff6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:14.131116 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a440ccd9eade4b3a9bc671b275b98ff6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:14.133093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:14.181749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a440ccd9eade4b3a9bc671b275b98ff6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.190026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a440ccd9eade4b3a9bc671b275b98ff6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.190674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a440ccd9eade4b3a9bc671b275b98ff6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.191317 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a440ccd9eade4b3a9bc671b275b98ff6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.198713 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a440ccd9eade4b3a9bc671b275b98ff6'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.216058 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1389/2763] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:14 2022] DELETE /identity/v3/users/a440ccd9eade4b3a9bc671b275b98ff6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:14.220876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7046cda-2416-4524-b76f-9894f8a2bb93 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:14.221376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7046cda-2416-4524-b76f-9894f8a2bb93 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:14.221478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7046cda-2416-4524-b76f-9894f8a2bb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:14.241056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-c7046cda-2416-4524-b76f-9894f8a2bb93 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:14.242240 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c7046cda-2416-4524-b76f-9894f8a2bb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:14.254318 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:14.301911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1375/2764] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:14.312206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:14.313614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:14.369943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1df568-bb47-40ea-a6c9-506719c26ab0 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:14.374467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8e7d4-84ca-405b-9089-691a0039e330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:14.375080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8e7d4-84ca-405b-9089-691a0039e330 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:14.375357 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8e7d4-84ca-405b-9089-691a0039e330 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:14.376367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc8e7d4-84ca-405b-9089-691a0039e330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:14.386095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8e7d4-84ca-405b-9089-691a0039e330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:14.387672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc8e7d4-84ca-405b-9089-691a0039e330 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:14.397758 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1390/2765] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:14.495567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7046cda-2416-4524-b76f-9894f8a2bb93 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:14.496996 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c7046cda-2416-4524-b76f-9894f8a2bb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:14.510706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7046cda-2416-4524-b76f-9894f8a2bb93 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:14.515944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37104347-f8db-4da0-a34a-092e7e435e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:14.516786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37104347-f8db-4da0-a34a-092e7e435e92 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:14.517074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37104347-f8db-4da0-a34a-092e7e435e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:14.517297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37104347-f8db-4da0-a34a-092e7e435e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:14.519590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-37104347-f8db-4da0-a34a-092e7e435e92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:14.533201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37104347-f8db-4da0-a34a-092e7e435e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:14.534623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37104347-f8db-4da0-a34a-092e7e435e92 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:14.545386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1376/2766] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:31:14.687740 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8e7d4-84ca-405b-9089-691a0039e330 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:14.689207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-dfc8e7d4-84ca-405b-9089-691a0039e330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:14.714529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc8e7d4-84ca-405b-9089-691a0039e330 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:14.721439 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:14.722141 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:14.722487 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:14.722829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] PATH_INFO: `/v3/projects/cde8896097b748078b927cf8a6f8aa43` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:14.724130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde8896097b748078b927cf8a6f8aa43)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:14.726137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:14.781975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:14.785300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde8896097b748078b927cf8a6f8aa43', 'request_id': 'req-cc0133d1-87f7-4325-91da-65296233a472'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.792872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde8896097b748078b927cf8a6f8aa43', 'request_id': 'req-cc0133d1-87f7-4325-91da-65296233a472'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.801355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde8896097b748078b927cf8a6f8aa43', 'request_id': 'req-cc0133d1-87f7-4325-91da-65296233a472'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.819602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] The token cache is being invalidate because project cde8896097b748078b927cf8a6f8aa43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:14.820114 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.821161 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.824164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1391/2767] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:14 2022] DELETE /identity/v3/projects/cde8896097b748078b927cf8a6f8aa43 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:14.829099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37104347-f8db-4da0-a34a-092e7e435e92 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:14.840949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37104347-f8db-4da0-a34a-092e7e435e92 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:14.907868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:14.908516 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:14.908799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:14.909080 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] PATH_INFO: `/v3/users/24ebcb6773c740ee98bdc44f16dbbe51` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:14.910227 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ebcb6773c740ee98bdc44f16dbbe51)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:14.912206 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:14.965346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ebcb6773c740ee98bdc44f16dbbe51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.966141 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ebcb6773c740ee98bdc44f16dbbe51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.975480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ebcb6773c740ee98bdc44f16dbbe51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.976732 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ebcb6773c740ee98bdc44f16dbbe51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:14.986093 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-595356ea-3df7-4d19-8b2d-65198e3253c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ebcb6773c740ee98bdc44f16dbbe51'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:15.004649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1377/2768] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:14 2022] DELETE /identity/v3/users/24ebcb6773c740ee98bdc44f16dbbe51 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:15.322795 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:15.324746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:15.350767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0133d1-87f7-4325-91da-65296233a472 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:15.412711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:15.413889 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:15.414173 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:15.414453 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] PATH_INFO: `/v3/projects/0ec3a0f6e94f4f90ae198ace742abbee` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:15.417258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec3a0f6e94f4f90ae198ace742abbee)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:15.419347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:15.472996 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec3a0f6e94f4f90ae198ace742abbee', 'request_id': 'req-dcf6d574-b061-4e2a-a270-ff7f7e064c91'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:15.479360 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec3a0f6e94f4f90ae198ace742abbee', 'request_id': 'req-dcf6d574-b061-4e2a-a270-ff7f7e064c91'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:15.487638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec3a0f6e94f4f90ae198ace742abbee', 'request_id': 'req-dcf6d574-b061-4e2a-a270-ff7f7e064c91'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:15.498672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] The token cache is being invalidate because project 0ec3a0f6e94f4f90ae198ace742abbee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:15.498958 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:15.499608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-dcf6d574-b061-4e2a-a270-ff7f7e064c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:15.501169 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1392/2769] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:15 2022] DELETE /identity/v3/projects/0ec3a0f6e94f4f90ae198ace742abbee => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:15.506489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58c282-b2fd-412a-bb3e-e831084694a2 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:15.506885 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58c282-b2fd-412a-bb3e-e831084694a2 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:15.507315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58c282-b2fd-412a-bb3e-e831084694a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:15.531360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-dc58c282-b2fd-412a-bb3e-e831084694a2 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:15.532864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-dc58c282-b2fd-412a-bb3e-e831084694a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:15.552336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:15.609722 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1378/2770] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:15.617334 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f79785-8b71-4cb3-b0af-6f0a544ad821 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:15.617700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f79785-8b71-4cb3-b0af-6f0a544ad821 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:15.618049 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f79785-8b71-4cb3-b0af-6f0a544ad821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:15.642787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-e9f79785-8b71-4cb3-b0af-6f0a544ad821 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:15.644224 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e9f79785-8b71-4cb3-b0af-6f0a544ad821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:15.652523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1393/2771] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:15.657412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58c282-b2fd-412a-bb3e-e831084694a2 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:15.667904 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58c282-b2fd-412a-bb3e-e831084694a2 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:15.673381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a269a4-c497-46d2-ae03-3e1189a13739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:15.673992 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a269a4-c497-46d2-ae03-3e1189a13739 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:15.674266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a269a4-c497-46d2-ae03-3e1189a13739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:15.674529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a269a4-c497-46d2-ae03-3e1189a13739 admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:15.675182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a269a4-c497-46d2-ae03-3e1189a13739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:31:15.675467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a269a4-c497-46d2-ae03-3e1189a13739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:15.677144 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a269a4-c497-46d2-ae03-3e1189a13739 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:15.689703 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1379/2772] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:31:15.694164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f79785-8b71-4cb3-b0af-6f0a544ad821 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:15.704019 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f79785-8b71-4cb3-b0af-6f0a544ad821 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:15.708663 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f7f01-c163-4145-9534-b0894b16ac5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:15.709294 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f7f01-c163-4145-9534-b0894b16ac5f admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:15.709571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f7f01-c163-4145-9534-b0894b16ac5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:15.709839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f7f01-c163-4145-9534-b0894b16ac5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:15.710481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f7f01-c163-4145-9534-b0894b16ac5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:15.712412 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f7f01-c163-4145-9534-b0894b16ac5f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:15.740763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1394/2773] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:15 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:15.746496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a269a4-c497-46d2-ae03-3e1189a13739 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:15.762659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a269a4-c497-46d2-ae03-3e1189a13739 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:15.830259 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1043a-4bee-49c1-b196-3dd10712dea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:15.830883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1043a-4bee-49c1-b196-3dd10712dea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:15.831160 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1043a-4bee-49c1-b196-3dd10712dea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:15.831430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1043a-4bee-49c1-b196-3dd10712dea4 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:15.832110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1043a-4bee-49c1-b196-3dd10712dea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:15.834089 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1043a-4bee-49c1-b196-3dd10712dea4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:15.863883 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1380/2774] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:15 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:15.865814 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f7f01-c163-4145-9534-b0894b16ac5f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:15.877976 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f7f01-c163-4145-9534-b0894b16ac5f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:15.884014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d334d66-3311-4f8b-8aab-0fecdd6eb064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:15.885466 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d334d66-3311-4f8b-8aab-0fecdd6eb064 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:15.886082 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d334d66-3311-4f8b-8aab-0fecdd6eb064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:15.886512 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d334d66-3311-4f8b-8aab-0fecdd6eb064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:15.887316 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d334d66-3311-4f8b-8aab-0fecdd6eb064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:15.890014 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d334d66-3311-4f8b-8aab-0fecdd6eb064 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:15.915365 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1395/2775] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:31:15.916196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1043a-4bee-49c1-b196-3dd10712dea4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:15.945058 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1043a-4bee-49c1-b196-3dd10712dea4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:15.950222 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01214984-8f71-49f5-9146-eae04bfb4fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:15.951479 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01214984-8f71-49f5-9146-eae04bfb4fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:15.951776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01214984-8f71-49f5-9146-eae04bfb4fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:15.952050 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01214984-8f71-49f5-9146-eae04bfb4fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:15.952731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01214984-8f71-49f5-9146-eae04bfb4fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:15.954323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01214984-8f71-49f5-9146-eae04bfb4fc4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:15.972143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1381/2776] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:15.973593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d334d66-3311-4f8b-8aab-0fecdd6eb064 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:15.984766 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d334d66-3311-4f8b-8aab-0fecdd6eb064 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:15.989767 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09948acf-55bd-4e53-bf94-75897539b811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:15.990516 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09948acf-55bd-4e53-bf94-75897539b811 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:15.990894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09948acf-55bd-4e53-bf94-75897539b811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:15.991229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09948acf-55bd-4e53-bf94-75897539b811 admin admin] PATH_INFO: `/v3/projects/268c4b5144dc4b89b6433997055a50b3/users/95828cf465384bf29f99d06591faa2ca/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:16.006865 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09948acf-55bd-4e53-bf94-75897539b811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268c4b5144dc4b89b6433997055a50b3, user_id=95828cf465384bf29f99d06591faa2ca, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:16.009595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09948acf-55bd-4e53-bf94-75897539b811 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:16.020130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:16.025800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1396/2777] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:15 2022] PUT /identity/v3/projects/268c4b5144dc4b89b6433997055a50b3/users/95828cf465384bf29f99d06591faa2ca/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:31:16.038000 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34da55-2313-4e55-9af9-11093a14fb44 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:16.038567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34da55-2313-4e55-9af9-11093a14fb44 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:16.038863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34da55-2313-4e55-9af9-11093a14fb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:16.067023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8e34da55-2313-4e55-9af9-11093a14fb44 None None] MFA Rules not processed for user `95828cf465384bf29f99d06591faa2ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:16.068327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e34da55-2313-4e55-9af9-11093a14fb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:16.136987 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1382/2778] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:31:16.149735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09948acf-55bd-4e53-bf94-75897539b811 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:16.151437 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-09948acf-55bd-4e53-bf94-75897539b811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:16.210072 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09948acf-55bd-4e53-bf94-75897539b811 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:16.213847 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f633d-4b9b-403d-9b53-01050cb4359f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56a94ccca15c462c9e7a5e88b907f420', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:16.214454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f633d-4b9b-403d-9b53-01050cb4359f service glance] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:16.214731 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f633d-4b9b-403d-9b53-01050cb4359f service glance] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:16.215002 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f633d-4b9b-403d-9b53-01050cb4359f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:16.223825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f633d-4b9b-403d-9b53-01050cb4359f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:16.225264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f633d-4b9b-403d-9b53-01050cb4359f service glance] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:16.235153 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1397/2779] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:17.947158 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34da55-2313-4e55-9af9-11093a14fb44 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:17.949977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8e34da55-2313-4e55-9af9-11093a14fb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:17.974759 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:18.022664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34da55-2313-4e55-9af9-11093a14fb44 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:18.027146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:18.027806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.028091 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.028365 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] PATH_INFO: `/v3/users/c950eb00b9474b29b86d24983e3b1583` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.030110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c950eb00b9474b29b86d24983e3b1583)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:18.032014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:18.083668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c950eb00b9474b29b86d24983e3b1583'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.084373 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c950eb00b9474b29b86d24983e3b1583'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.092430 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c950eb00b9474b29b86d24983e3b1583'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.093045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c950eb00b9474b29b86d24983e3b1583'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.103644 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-d6c0a377-e673-494e-bcf8-0e1f25fac2c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c950eb00b9474b29b86d24983e3b1583'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.124729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1383/2780] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:17 2022] DELETE /identity/v3/users/c950eb00b9474b29b86d24983e3b1583 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:18.312849 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f633d-4b9b-403d-9b53-01050cb4359f service glance] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:18.328306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f633d-4b9b-403d-9b53-01050cb4359f service glance] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:18.386796 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:18.387425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.387700 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.388004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] PATH_INFO: `/v3/projects/5e6ef4f3de1943ffbde82d3f0a19542f` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.390358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e6ef4f3de1943ffbde82d3f0a19542f)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:18.394264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:18.440629 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e6ef4f3de1943ffbde82d3f0a19542f', 'request_id': 'req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.446970 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e6ef4f3de1943ffbde82d3f0a19542f', 'request_id': 'req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.455781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e6ef4f3de1943ffbde82d3f0a19542f', 'request_id': 'req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.466840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] The token cache is being invalidate because project 5e6ef4f3de1943ffbde82d3f0a19542f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:18.467129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.467858 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5e8f1ad1-1afc-4af8-a27a-9ab02480dfa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:18.469440 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1398/2781] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:18 2022] DELETE /identity/v3/projects/5e6ef4f3de1943ffbde82d3f0a19542f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:18.474525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707ae67-b200-436f-bdef-8b0b3b7f6aab None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.474846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707ae67-b200-436f-bdef-8b0b3b7f6aab None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.475220 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707ae67-b200-436f-bdef-8b0b3b7f6aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.498876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-8707ae67-b200-436f-bdef-8b0b3b7f6aab None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:18.500130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8707ae67-b200-436f-bdef-8b0b3b7f6aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:18.572420 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1384/2782] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:18.580940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171167c5-cf0b-4d24-96f8-bf1e107ba060 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.581227 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171167c5-cf0b-4d24-96f8-bf1e107ba060 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.581507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171167c5-cf0b-4d24-96f8-bf1e107ba060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.601723 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-171167c5-cf0b-4d24-96f8-bf1e107ba060 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:18.602937 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-171167c5-cf0b-4d24-96f8-bf1e107ba060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:18.611373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1399/2783] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:18.615805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707ae67-b200-436f-bdef-8b0b3b7f6aab None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:18.626485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707ae67-b200-436f-bdef-8b0b3b7f6aab None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:18.631362 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5f354-4ece-4d51-a1a3-4905dd8f8fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:18.632015 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5f354-4ece-4d51-a1a3-4905dd8f8fae admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.632299 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5f354-4ece-4d51-a1a3-4905dd8f8fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.632573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5f354-4ece-4d51-a1a3-4905dd8f8fae admin admin] PATH_INFO: `/v3/domains` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.633223 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5f354-4ece-4d51-a1a3-4905dd8f8fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:31:18.633527 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5f354-4ece-4d51-a1a3-4905dd8f8fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:18.635129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5f354-4ece-4d51-a1a3-4905dd8f8fae admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:18.647319 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1385/2784] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:31:18.651872 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171167c5-cf0b-4d24-96f8-bf1e107ba060 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:18.662070 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171167c5-cf0b-4d24-96f8-bf1e107ba060 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:18.667077 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d11a6-7875-4446-a32e-246e5f30debf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:18.667743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d11a6-7875-4446-a32e-246e5f30debf admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.668056 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d11a6-7875-4446-a32e-246e5f30debf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.668410 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d11a6-7875-4446-a32e-246e5f30debf admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.669067 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d11a6-7875-4446-a32e-246e5f30debf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:18.671489 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d11a6-7875-4446-a32e-246e5f30debf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:18.699544 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1400/2785] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:18 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:18.703776 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5f354-4ece-4d51-a1a3-4905dd8f8fae admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:18.713769 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5f354-4ece-4d51-a1a3-4905dd8f8fae admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:18.737679 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:18.771655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c29d8a-1b20-4ce9-98f5-32f0b06767b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:18.772286 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c29d8a-1b20-4ce9-98f5-32f0b06767b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.772564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c29d8a-1b20-4ce9-98f5-32f0b06767b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.772831 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c29d8a-1b20-4ce9-98f5-32f0b06767b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.773448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c29d8a-1b20-4ce9-98f5-32f0b06767b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:18.775346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c29d8a-1b20-4ce9-98f5-32f0b06767b4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:18.796269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1386/2786] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:18 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:18.800449 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d11a6-7875-4446-a32e-246e5f30debf admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:18.807498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:18.812335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d11a6-7875-4446-a32e-246e5f30debf admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:18.818367 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1fb90-e41b-4005-9e5a-f17af60c165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:18.818984 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1fb90-e41b-4005-9e5a-f17af60c165c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.819564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1fb90-e41b-4005-9e5a-f17af60c165c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.819991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e1fb90-e41b-4005-9e5a-f17af60c165c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.820725 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1fb90-e41b-4005-9e5a-f17af60c165c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:18.822435 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e1fb90-e41b-4005-9e5a-f17af60c165c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:18.835458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1401/2787] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:31:18.839728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c29d8a-1b20-4ce9-98f5-32f0b06767b4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:18.854599 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c29d8a-1b20-4ce9-98f5-32f0b06767b4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:18.859806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49f010-accc-4a48-a94d-8a4a045291ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:18.860426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49f010-accc-4a48-a94d-8a4a045291ab admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.860717 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49f010-accc-4a48-a94d-8a4a045291ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.861121 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49f010-accc-4a48-a94d-8a4a045291ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.861653 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49f010-accc-4a48-a94d-8a4a045291ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:18.865743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49f010-accc-4a48-a94d-8a4a045291ab admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:18.875110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1387/2788] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:31:18.881755 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1fb90-e41b-4005-9e5a-f17af60c165c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:18.892392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e1fb90-e41b-4005-9e5a-f17af60c165c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:18.897039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b4ffb-a12a-4ee7-be2e-ab64c04dbc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:18.897692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b4ffb-a12a-4ee7-be2e-ab64c04dbc59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.897975 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b4ffb-a12a-4ee7-be2e-ab64c04dbc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.898251 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b4ffb-a12a-4ee7-be2e-ab64c04dbc59 admin admin] PATH_INFO: `/v3/projects/bd7f2ef25e054b619068baf878bead18/users/90d4dfe489634f3c9346b0ab86408e3b/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.913414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b4ffb-a12a-4ee7-be2e-ab64c04dbc59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7f2ef25e054b619068baf878bead18, user_id=90d4dfe489634f3c9346b0ab86408e3b, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:18.916168 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b4ffb-a12a-4ee7-be2e-ab64c04dbc59 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:18.929759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1402/2789] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:18 2022] PUT /identity/v3/projects/bd7f2ef25e054b619068baf878bead18/users/90d4dfe489634f3c9346b0ab86408e3b/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:18.934835 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48b868-ae70-4aff-99b2-f01b443540a7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:18.935124 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48b868-ae70-4aff-99b2-f01b443540a7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:18.935396 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48b868-ae70-4aff-99b2-f01b443540a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:18.954443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-7a48b868-ae70-4aff-99b2-f01b443540a7 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:18.955647 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-7a48b868-ae70-4aff-99b2-f01b443540a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:19.015930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1388/2790] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:19.025969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b4ffb-a12a-4ee7-be2e-ab64c04dbc59 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:19.027951 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ce7b4ffb-a12a-4ee7-be2e-ab64c04dbc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:19.085701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b4ffb-a12a-4ee7-be2e-ab64c04dbc59 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:19.089522 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642222d5-b08f-4a0e-bda6-5851bf672873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:19.090117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642222d5-b08f-4a0e-bda6-5851bf672873 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:19.090397 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642222d5-b08f-4a0e-bda6-5851bf672873 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:19.090672 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642222d5-b08f-4a0e-bda6-5851bf672873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:19.101136 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642222d5-b08f-4a0e-bda6-5851bf672873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:19.102539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642222d5-b08f-4a0e-bda6-5851bf672873 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:19.112560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1403/2791] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:23.505545 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c1cbd-a854-40bf-87d4-3b8c77b73a7d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:23.506042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c1cbd-a854-40bf-87d4-3b8c77b73a7d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:23.506144 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c1cbd-a854-40bf-87d4-3b8c77b73a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:23.553285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-3e6c1cbd-a854-40bf-87d4-3b8c77b73a7d None None] MFA Rules not processed for user `90d4dfe489634f3c9346b0ab86408e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:23.554511 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-3e6c1cbd-a854-40bf-87d4-3b8c77b73a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:23.630353 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1389/2792] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:23.647743 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642222d5-b08f-4a0e-bda6-5851bf672873 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:23.649392 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-642222d5-b08f-4a0e-bda6-5851bf672873 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:23.730355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642222d5-b08f-4a0e-bda6-5851bf672873 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:23.736577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8db36d-b70a-48b7-94fc-1214ddcf8a96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:23.737306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8db36d-b70a-48b7-94fc-1214ddcf8a96 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:23.737619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8db36d-b70a-48b7-94fc-1214ddcf8a96 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:23.737895 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8db36d-b70a-48b7-94fc-1214ddcf8a96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:23.747432 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8db36d-b70a-48b7-94fc-1214ddcf8a96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:23.749718 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8db36d-b70a-48b7-94fc-1214ddcf8a96 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:23.759574 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1404/2793] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:31:26.069279 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b08c4-7585-4b63-bf38-f96f7306c243 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:26.069746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b08c4-7585-4b63-bf38-f96f7306c243 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:26.069924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b08c4-7585-4b63-bf38-f96f7306c243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:26.092940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-530b08c4-7585-4b63-bf38-f96f7306c243 None None] MFA Rules not processed for user `90d4dfe489634f3c9346b0ab86408e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:26.094195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-530b08c4-7585-4b63-bf38-f96f7306c243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:26.101553 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1390/2794] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:33.755212 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=73,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:36.085863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:49.498904 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8db36d-b70a-48b7-94fc-1214ddcf8a96 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:49.500871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-ae8db36d-b70a-48b7-94fc-1214ddcf8a96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:49.514353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8db36d-b70a-48b7-94fc-1214ddcf8a96 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:49.519685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:49.520332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:49.520616 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:49.520888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] PATH_INFO: `/v3/users/95828cf465384bf29f99d06591faa2ca` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:49.522005 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95828cf465384bf29f99d06591faa2ca)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:49.523952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:49.571749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95828cf465384bf29f99d06591faa2ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:49.580091 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95828cf465384bf29f99d06591faa2ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:49.580704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95828cf465384bf29f99d06591faa2ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:49.581303 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95828cf465384bf29f99d06591faa2ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:49.588306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95828cf465384bf29f99d06591faa2ca'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:49.605768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1405/2795] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:31:49 2022] DELETE /identity/v3/users/95828cf465384bf29f99d06591faa2ca => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:49.610291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12a45-e29c-4974-b997-5252ac1fb01d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:49.610798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12a45-e29c-4974-b997-5252ac1fb01d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:49.610899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12a45-e29c-4974-b997-5252ac1fb01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:49.630143 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-21c12a45-e29c-4974-b997-5252ac1fb01d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:49.631387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-21c12a45-e29c-4974-b997-5252ac1fb01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:49.689418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1391/2796] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:31:49.699119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:49.700763 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:49.757389 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984febae-86f3-49a5-9e6c-5b6398a92178 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:49.760940 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56de00c-df4b-4bff-aa8b-d01ac253393f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:49.761546 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56de00c-df4b-4bff-aa8b-d01ac253393f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:49.761825 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56de00c-df4b-4bff-aa8b-d01ac253393f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:49.762100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56de00c-df4b-4bff-aa8b-d01ac253393f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:49.771193 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56de00c-df4b-4bff-aa8b-d01ac253393f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:49.772631 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56de00c-df4b-4bff-aa8b-d01ac253393f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:49.781997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1406/2797] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:50.047893 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12a45-e29c-4974-b997-5252ac1fb01d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:50.049863 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-21c12a45-e29c-4974-b997-5252ac1fb01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:50.061412 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12a45-e29c-4974-b997-5252ac1fb01d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:50.067103 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:50.067939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.068218 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.068490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] PATH_INFO: `/v3/projects/268c4b5144dc4b89b6433997055a50b3` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.069651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268c4b5144dc4b89b6433997055a50b3)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:50.072178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:50.122911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268c4b5144dc4b89b6433997055a50b3', 'request_id': 'req-94692428-bdca-4ee0-a152-ee40360f6294'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:50.130866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268c4b5144dc4b89b6433997055a50b3', 'request_id': 'req-94692428-bdca-4ee0-a152-ee40360f6294'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:50.141454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268c4b5144dc4b89b6433997055a50b3', 'request_id': 'req-94692428-bdca-4ee0-a152-ee40360f6294'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:50.154182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] The token cache is being invalidate because project 268c4b5144dc4b89b6433997055a50b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:31:50.154498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:50.155145 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-94692428-bdca-4ee0-a152-ee40360f6294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:31:50.157398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1392/2798] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:31:50 2022] DELETE /identity/v3/projects/268c4b5144dc4b89b6433997055a50b3 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:50.162304 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c8e4f-9d8a-4036-ba8e-e608174bf38a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.162591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c8e4f-9d8a-4036-ba8e-e608174bf38a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.162862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c8e4f-9d8a-4036-ba8e-e608174bf38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.184623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-448c8e4f-9d8a-4036-ba8e-e608174bf38a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:50.185829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-448c8e4f-9d8a-4036-ba8e-e608174bf38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:50.247404 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1407/2799] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:50.255375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b3a53-0544-431e-9fec-32fe2307a2b9 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.255745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b3a53-0544-431e-9fec-32fe2307a2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.256178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b3a53-0544-431e-9fec-32fe2307a2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.276191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9e9b3a53-0544-431e-9fec-32fe2307a2b9 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:50.277399 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9e9b3a53-0544-431e-9fec-32fe2307a2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:50.285983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1393/2800] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:50.290444 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c8e4f-9d8a-4036-ba8e-e608174bf38a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:50.301599 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c8e4f-9d8a-4036-ba8e-e608174bf38a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:50.306174 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bc2d1-b626-4db1-9a9d-1e3775b66730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:50.306804 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bc2d1-b626-4db1-9a9d-1e3775b66730 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.307132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bc2d1-b626-4db1-9a9d-1e3775b66730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.307400 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bc2d1-b626-4db1-9a9d-1e3775b66730 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.308068 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bc2d1-b626-4db1-9a9d-1e3775b66730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:31:50.308369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bc2d1-b626-4db1-9a9d-1e3775b66730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:50.309959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30bc2d1-b626-4db1-9a9d-1e3775b66730 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:50.322836 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1408/2801] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:31:50.327537 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b3a53-0544-431e-9fec-32fe2307a2b9 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:50.336571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b3a53-0544-431e-9fec-32fe2307a2b9 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:50.341150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4f691-0dfe-41d6-ae84-3eca418a8bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:50.341737 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4f691-0dfe-41d6-ae84-3eca418a8bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.342034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4f691-0dfe-41d6-ae84-3eca418a8bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.342298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4f691-0dfe-41d6-ae84-3eca418a8bf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.342916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4f691-0dfe-41d6-ae84-3eca418a8bf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:50.345341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4f691-0dfe-41d6-ae84-3eca418a8bf7 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:50.372489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1394/2802] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:50 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:31:50.376787 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bc2d1-b626-4db1-9a9d-1e3775b66730 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:50.382037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:50.388257 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30bc2d1-b626-4db1-9a9d-1e3775b66730 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:50.446681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d3984-30b4-4f90-99aa-d110ac1154fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:50.447298 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3984-30b4-4f90-99aa-d110ac1154fa admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.447584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3984-30b4-4f90-99aa-d110ac1154fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.447887 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d3984-30b4-4f90-99aa-d110ac1154fa admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.448511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d3984-30b4-4f90-99aa-d110ac1154fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:50.450647 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d3984-30b4-4f90-99aa-d110ac1154fa admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:50.471908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1409/2803] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:50 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:50.476016 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4f691-0dfe-41d6-ae84-3eca418a8bf7 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:50.486491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4f691-0dfe-41d6-ae84-3eca418a8bf7 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:50.491056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53fc77-f915-487e-ae30-e280b00d2223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:50.491656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53fc77-f915-487e-ae30-e280b00d2223 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.491970 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53fc77-f915-487e-ae30-e280b00d2223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.492244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53fc77-f915-487e-ae30-e280b00d2223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.492925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53fc77-f915-487e-ae30-e280b00d2223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:50.494497 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53fc77-f915-487e-ae30-e280b00d2223 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:50.504315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1395/2804] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:50.508205 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d3984-30b4-4f90-99aa-d110ac1154fa admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:50.519194 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d3984-30b4-4f90-99aa-d110ac1154fa admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:50.523898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4499314-148a-469a-a670-e7353112f2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:50.524496 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4499314-148a-469a-a670-e7353112f2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.524775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4499314-148a-469a-a670-e7353112f2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.525048 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4499314-148a-469a-a670-e7353112f2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.525711 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4499314-148a-469a-a670-e7353112f2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:50.527332 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4499314-148a-469a-a670-e7353112f2e4 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:50.538628 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1410/2805] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:50.542969 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53fc77-f915-487e-ae30-e280b00d2223 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:50.552752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53fc77-f915-487e-ae30-e280b00d2223 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:50.557322 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9e55c-0a48-4870-9508-5730aad702c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:50.558059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9e55c-0a48-4870-9508-5730aad702c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.558262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9e55c-0a48-4870-9508-5730aad702c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.558557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9e55c-0a48-4870-9508-5730aad702c5 admin admin] PATH_INFO: `/v3/projects/04bda0b59d3b4bfabf6f1b84fec0d6a6/users/f84520fd4b434d99ad0dab1a1dd3b033/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.573012 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9e55c-0a48-4870-9508-5730aad702c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04bda0b59d3b4bfabf6f1b84fec0d6a6, user_id=f84520fd4b434d99ad0dab1a1dd3b033, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:50.575926 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade9e55c-0a48-4870-9508-5730aad702c5 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:50.588626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1396/2806] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:50 2022] PUT /identity/v3/projects/04bda0b59d3b4bfabf6f1b84fec0d6a6/users/f84520fd4b434d99ad0dab1a1dd3b033/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:31:50.594456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f39af6-7fcb-42d1-9973-1b98c4a2e341 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.594811 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f39af6-7fcb-42d1-9973-1b98c4a2e341 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.595000 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f39af6-7fcb-42d1-9973-1b98c4a2e341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.614906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d2f39af6-7fcb-42d1-9973-1b98c4a2e341 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:50.616135 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d2f39af6-7fcb-42d1-9973-1b98c4a2e341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:50.673928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1411/2807] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:31:50.683574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9e55c-0a48-4870-9508-5730aad702c5 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:50.685528 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ade9e55c-0a48-4870-9508-5730aad702c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:50.707929 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:50.744884 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade9e55c-0a48-4870-9508-5730aad702c5 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:50.748626 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed08dc9-5e6b-4ec3-80be-5cf2d1c88019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:50.749230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed08dc9-5e6b-4ec3-80be-5cf2d1c88019 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:50.749508 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed08dc9-5e6b-4ec3-80be-5cf2d1c88019 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:50.749780 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed08dc9-5e6b-4ec3-80be-5cf2d1c88019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:50.759037 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed08dc9-5e6b-4ec3-80be-5cf2d1c88019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:50.760491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed08dc9-5e6b-4ec3-80be-5cf2d1c88019 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:50.770822 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1397/2808] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:51.629980 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f39af6-7fcb-42d1-9973-1b98c4a2e341 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:51.644042 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f39af6-7fcb-42d1-9973-1b98c4a2e341 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:51.647983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de935c79-2bb6-4b77-9874-15ddfffed7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:51.648896 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de935c79-2bb6-4b77-9874-15ddfffed7ad service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:51.649181 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de935c79-2bb6-4b77-9874-15ddfffed7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:51.649452 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de935c79-2bb6-4b77-9874-15ddfffed7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:51.651431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-de935c79-2bb6-4b77-9874-15ddfffed7ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:51.662564 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de935c79-2bb6-4b77-9874-15ddfffed7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:51.664347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de935c79-2bb6-4b77-9874-15ddfffed7ad service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:51.675586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1412/2809] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:31:55.383020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa4865-2152-4a14-8583-2ab70654e40d None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:55.383582 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa4865-2152-4a14-8583-2ab70654e40d None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:55.383701 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa4865-2152-4a14-8583-2ab70654e40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:55.424266 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-5daa4865-2152-4a14-8583-2ab70654e40d None None] MFA Rules not processed for user `f84520fd4b434d99ad0dab1a1dd3b033`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:31:55.425496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-5daa4865-2152-4a14-8583-2ab70654e40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:31:55.498691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1398/2810] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:31:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:31:55.503728 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de935c79-2bb6-4b77-9874-15ddfffed7ad service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:55.515241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de935c79-2bb6-4b77-9874-15ddfffed7ad service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:55.521655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c80ed4-f345-481e-9991-9af720c663ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:55.522197 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80ed4-f345-481e-9991-9af720c663ba admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:55.522472 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80ed4-f345-481e-9991-9af720c663ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:55.522744 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80ed4-f345-481e-9991-9af720c663ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:55.523400 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c80ed4-f345-481e-9991-9af720c663ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:55.525359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c80ed4-f345-481e-9991-9af720c663ba admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:55.556328 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1413/2811] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:31:55 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:31:55.560295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa4865-2152-4a14-8583-2ab70654e40d None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:55.572581 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa4865-2152-4a14-8583-2ab70654e40d None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:55.636505 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af39c54-3a72-4bbe-8b6c-6c50dd733a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:55.637129 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af39c54-3a72-4bbe-8b6c-6c50dd733a9b admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:55.637409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af39c54-3a72-4bbe-8b6c-6c50dd733a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:55.638261 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af39c54-3a72-4bbe-8b6c-6c50dd733a9b admin admin] PATH_INFO: `/v3/users` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:55.638398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af39c54-3a72-4bbe-8b6c-6c50dd733a9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:55.640271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af39c54-3a72-4bbe-8b6c-6c50dd733a9b admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:55.668611 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1399/2812] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:31:55 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 12:31:55.669909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c80ed4-f345-481e-9991-9af720c663ba admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:55.681278 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c80ed4-f345-481e-9991-9af720c663ba admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:55.686592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaa2ea-0f82-43ac-b88b-b19cca406ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:55.687208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaa2ea-0f82-43ac-b88b-b19cca406ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:55.687555 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaa2ea-0f82-43ac-b88b-b19cca406ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:55.688383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaa2ea-0f82-43ac-b88b-b19cca406ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:55.689006 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaa2ea-0f82-43ac-b88b-b19cca406ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:55.690608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaa2ea-0f82-43ac-b88b-b19cca406ba8 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:55.703815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1414/2813] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:55.706378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af39c54-3a72-4bbe-8b6c-6c50dd733a9b admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:55.717068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af39c54-3a72-4bbe-8b6c-6c50dd733a9b admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:55.722284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30831bba-0bc7-4451-9037-dc22ab9c9d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:55.722961 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30831bba-0bc7-4451-9037-dc22ab9c9d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:55.723238 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30831bba-0bc7-4451-9037-dc22ab9c9d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:55.723509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30831bba-0bc7-4451-9037-dc22ab9c9d75 admin admin] PATH_INFO: `/v3/projects/aee7cc95fa89440a9d6e270a694f04b0/users/ddbcf7b849554d6b9b2dd4f497e4b6fa/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:55.740499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30831bba-0bc7-4451-9037-dc22ab9c9d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee7cc95fa89440a9d6e270a694f04b0, user_id=ddbcf7b849554d6b9b2dd4f497e4b6fa, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:55.743801 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30831bba-0bc7-4451-9037-dc22ab9c9d75 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:55.758846 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1400/2814] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:55 2022] PUT /identity/v3/projects/aee7cc95fa89440a9d6e270a694f04b0/users/ddbcf7b849554d6b9b2dd4f497e4b6fa/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:31:55.763450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaa2ea-0f82-43ac-b88b-b19cca406ba8 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:55.791592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaa2ea-0f82-43ac-b88b-b19cca406ba8 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:55.829618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:31:55.862764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810927f2-2091-4bd6-b7d0-0fb0653ae9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:55.863482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810927f2-2091-4bd6-b7d0-0fb0653ae9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:55.863855 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810927f2-2091-4bd6-b7d0-0fb0653ae9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:55.864197 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810927f2-2091-4bd6-b7d0-0fb0653ae9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:55.864914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810927f2-2091-4bd6-b7d0-0fb0653ae9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:55.866618 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810927f2-2091-4bd6-b7d0-0fb0653ae9bf admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:55.878134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1415/2815] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:31:55.882289 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30831bba-0bc7-4451-9037-dc22ab9c9d75 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:31:55.893674 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30831bba-0bc7-4451-9037-dc22ab9c9d75 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:31:55.899448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e9a6d-e446-4835-9a65-b33e2bae5b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:31:55.900253 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e9a6d-e446-4835-9a65-b33e2bae5b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:31:55.900538 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e9a6d-e446-4835-9a65-b33e2bae5b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:31:55.900815 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e9a6d-e446-4835-9a65-b33e2bae5b50 admin admin] PATH_INFO: `/v3/projects/aee7cc95fa89440a9d6e270a694f04b0/users/ddbcf7b849554d6b9b2dd4f497e4b6fa/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:31:55.902774 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e9a6d-e446-4835-9a65-b33e2bae5b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee7cc95fa89440a9d6e270a694f04b0, user_id=ddbcf7b849554d6b9b2dd4f497e4b6fa, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:31:55.905812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e9a6d-e446-4835-9a65-b33e2bae5b50 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:31:55.918889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1401/2816] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:31:55 2022] PUT /identity/v3/projects/aee7cc95fa89440a9d6e270a694f04b0/users/ddbcf7b849554d6b9b2dd4f497e4b6fa/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:32:00.553827 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593da884-23ba-4769-ab9f-055a4502920e None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:00.554306 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593da884-23ba-4769-ab9f-055a4502920e None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:00.554403 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593da884-23ba-4769-ab9f-055a4502920e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:00.587349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-593da884-23ba-4769-ab9f-055a4502920e None None] MFA Rules not processed for user `ddbcf7b849554d6b9b2dd4f497e4b6fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:32:00.588622 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-593da884-23ba-4769-ab9f-055a4502920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:00.660800 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1416/2817] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:32:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:32:00.677244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e9a6d-e446-4835-9a65-b33e2bae5b50 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:32:00.679021 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-ca5e9a6d-e446-4835-9a65-b33e2bae5b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:00.686063 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:32:00.787252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e9a6d-e446-4835-9a65-b33e2bae5b50 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:32:00.792934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26776f8c-222a-4057-9b96-b64b7d0f121a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:32:00.793811 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26776f8c-222a-4057-9b96-b64b7d0f121a service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:00.794151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26776f8c-222a-4057-9b96-b64b7d0f121a service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:00.794490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26776f8c-222a-4057-9b96-b64b7d0f121a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:00.881378 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26776f8c-222a-4057-9b96-b64b7d0f121a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:32:00.882998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26776f8c-222a-4057-9b96-b64b7d0f121a service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:32:00.894094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1402/2818] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:32:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:32:10.657613 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=47 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:32:10.889019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=54 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:32:11.394442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593da884-23ba-4769-ab9f-055a4502920e None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:32:11.457362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593da884-23ba-4769-ab9f-055a4502920e None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:32:11.461347 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027b9a5-a656-4920-8c79-eb4dfa6b485f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:32:11.461972 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027b9a5-a656-4920-8c79-eb4dfa6b485f service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:11.462252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027b9a5-a656-4920-8c79-eb4dfa6b485f service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:11.462571 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027b9a5-a656-4920-8c79-eb4dfa6b485f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:11.464773 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-9027b9a5-a656-4920-8c79-eb4dfa6b485f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:11.475138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027b9a5-a656-4920-8c79-eb4dfa6b485f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:32:11.476735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027b9a5-a656-4920-8c79-eb4dfa6b485f service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:32:11.485291 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1417/2819] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:32:12.596620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26776f8c-222a-4057-9b96-b64b7d0f121a service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:32:12.606034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26776f8c-222a-4057-9b96-b64b7d0f121a service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:32:12.609867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5efdb-2137-4ecb-9384-0f4a2ffa711f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:32:12.610474 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5efdb-2137-4ecb-9384-0f4a2ffa711f service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:12.610841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5efdb-2137-4ecb-9384-0f4a2ffa711f service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:12.611178 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5efdb-2137-4ecb-9384-0f4a2ffa711f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:12.613001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-c8a5efdb-2137-4ecb-9384-0f4a2ffa711f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:12.626746 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5efdb-2137-4ecb-9384-0f4a2ffa711f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:32:12.628750 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5efdb-2137-4ecb-9384-0f4a2ffa711f service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:32:12.640161 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1403/2820] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:32:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:32:21.481497 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=26 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:32:22.634977 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:32:23.717543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97089e43-de72-4b33-8c28-a6f942385f57 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:23.718081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97089e43-de72-4b33-8c28-a6f942385f57 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:23.718081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97089e43-de72-4b33-8c28-a6f942385f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:23.737288 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-97089e43-de72-4b33-8c28-a6f942385f57 None None] MFA Rules not processed for user `f84520fd4b434d99ad0dab1a1dd3b033`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:32:23.738536 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97089e43-de72-4b33-8c28-a6f942385f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:23.746223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1418/2821] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:32:24.062200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5efdb-2137-4ecb-9384-0f4a2ffa711f service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:32:24.070752 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5efdb-2137-4ecb-9384-0f4a2ffa711f service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:32:24.074295 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4e708-82ad-41e0-a841-693257cb3d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:32:24.074882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4e708-82ad-41e0-a841-693257cb3d01 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:24.075150 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4e708-82ad-41e0-a841-693257cb3d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:24.075419 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4e708-82ad-41e0-a841-693257cb3d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:24.077387 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-e0b4e708-82ad-41e0-a841-693257cb3d01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:24.087198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4e708-82ad-41e0-a841-693257cb3d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:32:24.088622 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4e708-82ad-41e0-a841-693257cb3d01 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:32:24.096685 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1404/2822] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:32:24.544907 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97089e43-de72-4b33-8c28-a6f942385f57 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:32:24.546319 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-97089e43-de72-4b33-8c28-a6f942385f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:24.627076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97089e43-de72-4b33-8c28-a6f942385f57 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:32:24.632737 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:32:24.633359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:24.633649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:24.633926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:24.635776 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:24.648580 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:32:24.650175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:32:24.660302 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1419/2823] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:32:28.171568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4e708-82ad-41e0-a841-693257cb3d01 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:32:28.188294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4e708-82ad-41e0-a841-693257cb3d01 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:32:28.195294 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc5bca-62ab-43e0-b066-7c5aecd1dbaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:32:28.196359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc5bca-62ab-43e0-b066-7c5aecd1dbaa service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:28.196793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc5bca-62ab-43e0-b066-7c5aecd1dbaa service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:28.197212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc5bca-62ab-43e0-b066-7c5aecd1dbaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:28.200139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-95bc5bca-62ab-43e0-b066-7c5aecd1dbaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:28.226960 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc5bca-62ab-43e0-b066-7c5aecd1dbaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:32:28.229965 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc5bca-62ab-43e0-b066-7c5aecd1dbaa service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:32:28.242868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1405/2824] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:32:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:32:34.655918 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=35 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:32:38.236890 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=12 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:32:40.345043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:32:40.346529 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:40.361950 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a4601-bc4a-4b82-a5c6-1039a4e55a88 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:32:40.368007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b3e67-e69f-476c-b924-fdd0c3b3775d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:32:40.368614 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b3e67-e69f-476c-b924-fdd0c3b3775d service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:40.368892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b3e67-e69f-476c-b924-fdd0c3b3775d service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:40.369164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b3e67-e69f-476c-b924-fdd0c3b3775d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:40.370906 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-958b3e67-e69f-476c-b924-fdd0c3b3775d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:32:40.384167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b3e67-e69f-476c-b924-fdd0c3b3775d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:32:40.385759 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b3e67-e69f-476c-b924-fdd0c3b3775d service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:32:40.396583 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1420/2825] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:32:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:32:43.730370 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc5bca-62ab-43e0-b066-7c5aecd1dbaa service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:32:43.739729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc5bca-62ab-43e0-b066-7c5aecd1dbaa service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:32:43.743289 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235b03f-3393-4a37-b80d-6d45ba54c186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:32:43.743912 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235b03f-3393-4a37-b80d-6d45ba54c186 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:32:43.744189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235b03f-3393-4a37-b80d-6d45ba54c186 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:32:43.744454 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235b03f-3393-4a37-b80d-6d45ba54c186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:32:43.753494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235b03f-3393-4a37-b80d-6d45ba54c186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:32:43.754891 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235b03f-3393-4a37-b80d-6d45ba54c186 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:32:43.764934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1406/2826] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:32:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:32:50.392040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:32:53.760624 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:33:05.449183 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5323e-6a63-4820-9d22-49d0b15de900 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:33:05.449756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5323e-6a63-4820-9d22-49d0b15de900 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:33:05.449756 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5323e-6a63-4820-9d22-49d0b15de900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:33:05.469905 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f8d5323e-6a63-4820-9d22-49d0b15de900 None None] MFA Rules not processed for user `90d4dfe489634f3c9346b0ab86408e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:33:05.471134 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f8d5323e-6a63-4820-9d22-49d0b15de900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:33:05.529922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1421/2827] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:33:15.523189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=25 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:34:08.258693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235b03f-3393-4a37-b80d-6d45ba54c186 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:34:08.267953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235b03f-3393-4a37-b80d-6d45ba54c186 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:34:08.271720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd4201-f82d-4ec6-849f-05f049f64c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:34:08.272347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd4201-f82d-4ec6-849f-05f049f64c94 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:34:08.272629 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd4201-f82d-4ec6-849f-05f049f64c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:34:08.272901 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd4201-f82d-4ec6-849f-05f049f64c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:34:08.274735 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-a0dd4201-f82d-4ec6-849f-05f049f64c94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:34:08.333531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd4201-f82d-4ec6-849f-05f049f64c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:34:08.334960 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dd4201-f82d-4ec6-849f-05f049f64c94 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:34:08.344542 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1407/2828] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:34:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:34:12.312004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5323e-6a63-4820-9d22-49d0b15de900 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:34:12.323841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5323e-6a63-4820-9d22-49d0b15de900 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:34:12.328745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47af24-4db7-4443-a963-38a99907ddf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:34:12.329345 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47af24-4db7-4443-a963-38a99907ddf9 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:34:12.329649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47af24-4db7-4443-a963-38a99907ddf9 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:34:12.329926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47af24-4db7-4443-a963-38a99907ddf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:34:12.331656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4f47af24-4db7-4443-a963-38a99907ddf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:34:12.344477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47af24-4db7-4443-a963-38a99907ddf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:34:12.346100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47af24-4db7-4443-a963-38a99907ddf9 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:34:12.356842 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1422/2829] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:34:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:34:14.207355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd4201-f82d-4ec6-849f-05f049f64c94 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:34:14.217443 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dd4201-f82d-4ec6-849f-05f049f64c94 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:34:14.221077 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:34:14.221704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:34:14.221983 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:34:14.222256 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:34:14.224019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:34:14.287340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:34:14.288715 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:34:14.298549 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1408/2830] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:34:17.414864 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47af24-4db7-4443-a963-38a99907ddf9 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:34:17.428065 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47af24-4db7-4443-a963-38a99907ddf9 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:34:17.433226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e2f06-0e1e-4e7c-b672-4d83e5b0b753 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:34:17.434026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e2f06-0e1e-4e7c-b672-4d83e5b0b753 service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:34:17.434371 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e2f06-0e1e-4e7c-b672-4d83e5b0b753 service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:34:17.434714 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e2f06-0e1e-4e7c-b672-4d83e5b0b753 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:34:17.436572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e16e2f06-0e1e-4e7c-b672-4d83e5b0b753 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:34:17.448370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e2f06-0e1e-4e7c-b672-4d83e5b0b753 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:34:17.450534 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e2f06-0e1e-4e7c-b672-4d83e5b0b753 service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:34:17.461968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1423/2831] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:34:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:34:20.710466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:34:20.711932 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:34:20.725105 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d36ad-ba2b-4345-9988-ec05a07eeb12 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:34:20.729764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:34:20.730381 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:34:20.730660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:34:20.730930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] PATH_INFO: `/v3/users/94c4cdfb0f264398a7d3b155a9523b11` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:34:20.732202 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94c4cdfb0f264398a7d3b155a9523b11)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:34:20.734098 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:34:20.783994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94c4cdfb0f264398a7d3b155a9523b11'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:20.784643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94c4cdfb0f264398a7d3b155a9523b11'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:20.792068 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94c4cdfb0f264398a7d3b155a9523b11'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:20.792705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94c4cdfb0f264398a7d3b155a9523b11'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:20.799623 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94c4cdfb0f264398a7d3b155a9523b11'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:20.817198 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1409/2832] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:34:20 2022] DELETE /identity/v3/users/94c4cdfb0f264398a7d3b155a9523b11 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:34:20.993355 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e2f06-0e1e-4e7c-b672-4d83e5b0b753 service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:34:21.004637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e2f06-0e1e-4e7c-b672-4d83e5b0b753 service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:34:21.062735 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:34:21.063364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:34:21.063650 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:34:21.063949 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] PATH_INFO: `/v3/projects/95515a2858124583ac138aa6350e4bc3` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:34:21.065070 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] RBAC: Authorizing `identity:delete_project(project_id=95515a2858124583ac138aa6350e4bc3)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:34:21.066968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:34:21.112608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95515a2858124583ac138aa6350e4bc3', 'request_id': 'req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.118873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95515a2858124583ac138aa6350e4bc3', 'request_id': 'req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.127130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95515a2858124583ac138aa6350e4bc3', 'request_id': 'req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.137295 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] The token cache is being invalidate because project 95515a2858124583ac138aa6350e4bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:34:21.137576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.138189 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.139655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1424/2833] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:34:20 2022] DELETE /identity/v3/projects/95515a2858124583ac138aa6350e4bc3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:34:21.143959 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:34:21.145356 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:34:21.154838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:34:21.158869 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb7a5b-5bb2-4640-b5e2-9f86e9e5a2e4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:34:21.216583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:34:21.217130 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:34:21.217409 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:34:21.217682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] PATH_INFO: `/v3/users/28bb5d5441e8430a97ff57f2d375d363` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:34:21.218823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28bb5d5441e8430a97ff57f2d375d363)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:34:21.220799 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:34:21.267147 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28bb5d5441e8430a97ff57f2d375d363'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.267841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28bb5d5441e8430a97ff57f2d375d363'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.275306 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28bb5d5441e8430a97ff57f2d375d363'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.275939 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28bb5d5441e8430a97ff57f2d375d363'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.282880 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-3b8d7772-4f7e-44b6-a1ca-1c74b18f61c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28bb5d5441e8430a97ff57f2d375d363'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.300341 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1410/2834] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:34:21 2022] DELETE /identity/v3/users/28bb5d5441e8430a97ff57f2d375d363 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:34:21.479480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:34:21.481485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:34:21.494481 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b0b78-d7e8-4c19-9eb9-89b37efee94c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:34:21.558577 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:34:21.559211 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:34:21.559484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:34:21.559777 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] PATH_INFO: `/v3/projects/43c694babd814ea8bf1cdaf84f242a26` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:34:21.560892 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c694babd814ea8bf1cdaf84f242a26)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:34:21.562782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:34:21.569158 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:34:21.612661 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c694babd814ea8bf1cdaf84f242a26', 'request_id': 'req-09d4df4a-1140-4269-ba92-dac87b524cec'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.619491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c694babd814ea8bf1cdaf84f242a26', 'request_id': 'req-09d4df4a-1140-4269-ba92-dac87b524cec'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.627095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c694babd814ea8bf1cdaf84f242a26', 'request_id': 'req-09d4df4a-1140-4269-ba92-dac87b524cec'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.637669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] The token cache is being invalidate because project 43c694babd814ea8bf1cdaf84f242a26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:34:21.637979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.638667 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-09d4df4a-1140-4269-ba92-dac87b524cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:34:21.640226 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1425/2835] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:34:21 2022] DELETE /identity/v3/projects/43c694babd814ea8bf1cdaf84f242a26 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:34:31.296982 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:34:31.634509 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:34:45.257586 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113aaaf-e649-46cd-b121-f9194f4c3a2b None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:34:45.258296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113aaaf-e649-46cd-b121-f9194f4c3a2b None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:34:45.258296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113aaaf-e649-46cd-b121-f9194f4c3a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:34:45.279330 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d113aaaf-e649-46cd-b121-f9194f4c3a2b None None] MFA Rules not processed for user `90d4dfe489634f3c9346b0ab86408e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:34:45.280566 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d113aaaf-e649-46cd-b121-f9194f4c3a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:34:45.335181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1411/2836] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:34:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:34:55.328433 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=25 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:35:07.348372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e8e3f-07e7-4c01-888b-6f21790c319f None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:35:07.348922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e8e3f-07e7-4c01-888b-6f21790c319f None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:35:07.348922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e8e3f-07e7-4c01-888b-6f21790c319f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:35:07.370692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-378e8e3f-07e7-4c01-888b-6f21790c319f None None] MFA Rules not processed for user `f84520fd4b434d99ad0dab1a1dd3b033`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:35:07.372420 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-378e8e3f-07e7-4c01-888b-6f21790c319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:35:07.426413 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1426/2837] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:35:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:35:07.735584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113aaaf-e649-46cd-b121-f9194f4c3a2b None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:35:07.737049 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d113aaaf-e649-46cd-b121-f9194f4c3a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:35:07.794288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113aaaf-e649-46cd-b121-f9194f4c3a2b None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:35:07.797794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:35:07.798392 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:35:07.798667 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:35:07.798934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:35:07.800635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:35:07.810973 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:35:07.812424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:35:07.820291 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1412/2838] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:35:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:35:17.419704 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=25 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:35:17.816720 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=26 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:35:28.200745 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e8e3f-07e7-4c01-888b-6f21790c319f None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:35:28.202386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-378e8e3f-07e7-4c01-888b-6f21790c319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:35:28.277685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e8e3f-07e7-4c01-888b-6f21790c319f None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:35:28.283119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140322d7-1ef2-4bae-b36f-63ca03c7782f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:35:28.283736 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140322d7-1ef2-4bae-b36f-63ca03c7782f service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:35:28.284036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140322d7-1ef2-4bae-b36f-63ca03c7782f service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:35:28.284310 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140322d7-1ef2-4bae-b36f-63ca03c7782f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:35:28.286029 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-140322d7-1ef2-4bae-b36f-63ca03c7782f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:35:28.298775 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140322d7-1ef2-4bae-b36f-63ca03c7782f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:35:28.300430 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140322d7-1ef2-4bae-b36f-63ca03c7782f service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:35:28.312271 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1427/2839] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:35:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:35:38.307986 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=34 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:36:12.744476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:36:12.745923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:36:12.801671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cff441-1da7-4c19-bd1b-9fafc5d5f869 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:36:12.805262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5c6a0-afb4-4681-b410-d4daff322c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:36:12.805855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5c6a0-afb4-4681-b410-d4daff322c83 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:36:12.806197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5c6a0-afb4-4681-b410-d4daff322c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:36:12.806401 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5c6a0-afb4-4681-b410-d4daff322c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:36:12.808168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-acc5c6a0-afb4-4681-b410-d4daff322c83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:36:12.821350 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5c6a0-afb4-4681-b410-d4daff322c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:36:12.822754 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5c6a0-afb4-4681-b410-d4daff322c83 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:36:12.833933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1413/2840] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:36:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:36:13.001330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140322d7-1ef2-4bae-b36f-63ca03c7782f service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:36:13.010550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140322d7-1ef2-4bae-b36f-63ca03c7782f service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:36:13.014547 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8e032-c564-41f7-8f89-a18c899ff35e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:36:13.015146 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8e032-c564-41f7-8f89-a18c899ff35e service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:36:13.015424 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8e032-c564-41f7-8f89-a18c899ff35e service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:36:13.015697 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8e032-c564-41f7-8f89-a18c899ff35e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:36:13.017490 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e2e8e032-c564-41f7-8f89-a18c899ff35e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:36:13.031666 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8e032-c564-41f7-8f89-a18c899ff35e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:36:13.033122 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8e032-c564-41f7-8f89-a18c899ff35e service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:36:13.044630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1428/2841] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:36:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:36:19.905180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79922fd-9593-4256-8bd5-48f4b42eb118 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:36:19.905671 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79922fd-9593-4256-8bd5-48f4b42eb118 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:36:19.905793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79922fd-9593-4256-8bd5-48f4b42eb118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:36:19.924793 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f79922fd-9593-4256-8bd5-48f4b42eb118 None None] MFA Rules not processed for user `90d4dfe489634f3c9346b0ab86408e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:36:19.927074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f79922fd-9593-4256-8bd5-48f4b42eb118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:36:19.935427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1414/2842] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:36:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:36:22.751862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8e032-c564-41f7-8f89-a18c899ff35e service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:36:22.764269 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8e032-c564-41f7-8f89-a18c899ff35e service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:36:22.771701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a4ec2-02d9-4f9a-a0fe-bad11c21c735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:36:22.772348 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4ec2-02d9-4f9a-a0fe-bad11c21c735 service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:36:22.772627 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4ec2-02d9-4f9a-a0fe-bad11c21c735 service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:36:22.773094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4ec2-02d9-4f9a-a0fe-bad11c21c735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:36:22.774886 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-213a4ec2-02d9-4f9a-a0fe-bad11c21c735 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:36:22.785880 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a4ec2-02d9-4f9a-a0fe-bad11c21c735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:36:22.787479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a4ec2-02d9-4f9a-a0fe-bad11c21c735 service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:36:22.797377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1429/2843] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:36:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:36:29.919062 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=29 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:36:32.793576 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=12 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:36:57.569787 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79922fd-9593-4256-8bd5-48f4b42eb118 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:36:57.571602 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f79922fd-9593-4256-8bd5-48f4b42eb118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:36:57.585059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79922fd-9593-4256-8bd5-48f4b42eb118 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:36:57.588864 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c0835-54e8-4bfd-be42-b39db5fb90a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:36:57.589456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c0835-54e8-4bfd-be42-b39db5fb90a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:36:57.589731 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c0835-54e8-4bfd-be42-b39db5fb90a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:36:57.589998 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c0835-54e8-4bfd-be42-b39db5fb90a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:36:57.591867 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-bb3c0835-54e8-4bfd-be42-b39db5fb90a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:36:57.602281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c0835-54e8-4bfd-be42-b39db5fb90a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:36:57.603697 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c0835-54e8-4bfd-be42-b39db5fb90a4 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:36:57.611665 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1415/2844] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:36:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:37:07.608296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:37:18.837562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a4ec2-02d9-4f9a-a0fe-bad11c21c735 service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:37:18.847075 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a4ec2-02d9-4f9a-a0fe-bad11c21c735 service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:37:18.851010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76572b60-3780-4856-a84b-ecbd2bcc3e78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:37:18.851609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76572b60-3780-4856-a84b-ecbd2bcc3e78 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:37:18.851926 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76572b60-3780-4856-a84b-ecbd2bcc3e78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:37:18.852202 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76572b60-3780-4856-a84b-ecbd2bcc3e78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:37:18.853942 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-76572b60-3780-4856-a84b-ecbd2bcc3e78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:37:18.865780 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76572b60-3780-4856-a84b-ecbd2bcc3e78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:37:18.867206 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76572b60-3780-4856-a84b-ecbd2bcc3e78 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:37:18.877250 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1430/2845] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:37:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:37:28.872434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=6 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:37:28.895334 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f756f-06b2-47e2-9f6f-7cb8bf09d2ba None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:37:28.896386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f756f-06b2-47e2-9f6f-7cb8bf09d2ba None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:37:28.896386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f756f-06b2-47e2-9f6f-7cb8bf09d2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:37:28.915889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d36f756f-06b2-47e2-9f6f-7cb8bf09d2ba None None] MFA Rules not processed for user `f84520fd4b434d99ad0dab1a1dd3b033`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:37:28.917126 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d36f756f-06b2-47e2-9f6f-7cb8bf09d2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:37:28.924001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1416/2846] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:37:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:37:29.244385 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76572b60-3780-4856-a84b-ecbd2bcc3e78 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:37:29.253619 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76572b60-3780-4856-a84b-ecbd2bcc3e78 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:37:29.257788 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:37:29.258378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:37:29.258657 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:37:29.258930 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:37:29.260662 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:37:29.270191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:37:29.272234 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:37:29.280139 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1431/2847] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:37:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:37:30.732196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f756f-06b2-47e2-9f6f-7cb8bf09d2ba None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:37:30.733659 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d36f756f-06b2-47e2-9f6f-7cb8bf09d2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:37:30.804933 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f756f-06b2-47e2-9f6f-7cb8bf09d2ba None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:37:30.810079 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb65683-d0f5-493f-a598-ed6582bc3374 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:37:30.810689 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb65683-d0f5-493f-a598-ed6582bc3374 service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:37:30.810972 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb65683-d0f5-493f-a598-ed6582bc3374 service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:37:30.811247 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb65683-d0f5-493f-a598-ed6582bc3374 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:37:30.813179 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-eeb65683-d0f5-493f-a598-ed6582bc3374 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:37:30.825090 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb65683-d0f5-493f-a598-ed6582bc3374 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:37:30.826682 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb65683-d0f5-493f-a598-ed6582bc3374 service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:37:30.837889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1417/2848] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:37:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:37:36.646770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:37:36.648370 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:37:36.666590 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f58c87-edc1-4c8d-8b41-57de576db15e service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:37:36.672285 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144aef4-b594-480b-b76c-f9a5b85760fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:37:36.672993 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144aef4-b594-480b-b76c-f9a5b85760fa service nova] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:37:36.673330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144aef4-b594-480b-b76c-f9a5b85760fa service nova] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:37:36.673867 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144aef4-b594-480b-b76c-f9a5b85760fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:37:36.675873 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f144aef4-b594-480b-b76c-f9a5b85760fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:37:36.690378 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144aef4-b594-480b-b76c-f9a5b85760fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:37:36.692344 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144aef4-b594-480b-b76c-f9a5b85760fa service nova] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:37:36.705784 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1432/2849] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:37:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:37:40.832536 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=35 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:37:46.701088 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=12 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:37:47.846907 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb65683-d0f5-493f-a598-ed6582bc3374 service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:37:47.863734 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb65683-d0f5-493f-a598-ed6582bc3374 service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:37:47.874493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e3c7d-5510-4c20-a979-ee5aee6f027d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:37:47.875525 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e3c7d-5510-4c20-a979-ee5aee6f027d service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:37:47.876041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e3c7d-5510-4c20-a979-ee5aee6f027d service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:37:47.876509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e3c7d-5510-4c20-a979-ee5aee6f027d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:37:47.879473 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-556e3c7d-5510-4c20-a979-ee5aee6f027d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:37:47.894938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e3c7d-5510-4c20-a979-ee5aee6f027d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:37:47.897203 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e3c7d-5510-4c20-a979-ee5aee6f027d service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:37:47.908786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1418/2850] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:37:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:37:50.766491 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144aef4-b594-480b-b76c-f9a5b85760fa service nova] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:37:50.776377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144aef4-b594-480b-b76c-f9a5b85760fa service nova] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:37:50.780010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:37:50.780606 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:37:50.780888 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:37:50.781164 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:37:50.782925 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:37:50.846120 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:37:50.847552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:37:50.857553 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1433/2851] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:37:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:37:57.903707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=6 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:00.853218 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=30 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:06.398138 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e3c7d-5510-4c20-a979-ee5aee6f027d service nova] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:06.407660 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e3c7d-5510-4c20-a979-ee5aee6f027d service nova] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:06.411252 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8be5ae-6fff-4f14-970f-37a911d28c7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:06.411887 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8be5ae-6fff-4f14-970f-37a911d28c7f service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:06.412172 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8be5ae-6fff-4f14-970f-37a911d28c7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:06.412446 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8be5ae-6fff-4f14-970f-37a911d28c7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:06.414594 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-be8be5ae-6fff-4f14-970f-37a911d28c7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:06.482336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8be5ae-6fff-4f14-970f-37a911d28c7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:06.483782 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8be5ae-6fff-4f14-970f-37a911d28c7f service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:06.493838 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1419/2852] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:38:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:38:09.602793 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:09.604095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:09.619674 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c803a1c-5ea4-48f7-a5a8-9dfb674f6c84 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:09.624221 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:09.624841 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:09.625117 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:09.625384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] PATH_INFO: `/v3/users/90d4dfe489634f3c9346b0ab86408e3b` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:09.626507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=90d4dfe489634f3c9346b0ab86408e3b)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:09.628450 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:09.675947 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90d4dfe489634f3c9346b0ab86408e3b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:09.684423 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90d4dfe489634f3c9346b0ab86408e3b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:09.685207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90d4dfe489634f3c9346b0ab86408e3b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:09.686379 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90d4dfe489634f3c9346b0ab86408e3b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:09.694654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a99d14af-c584-47cc-a0b2-2906ea13a6ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90d4dfe489634f3c9346b0ab86408e3b'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:09.712445 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1434/2853] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:38:09 2022] DELETE /identity/v3/users/90d4dfe489634f3c9346b0ab86408e3b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:09.894531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8be5ae-6fff-4f14-970f-37a911d28c7f service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:09.905333 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8be5ae-6fff-4f14-970f-37a911d28c7f service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:09.961930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:09.962492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:09.962768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:09.963035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] PATH_INFO: `/v3/projects/bd7f2ef25e054b619068baf878bead18` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:09.964407 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7f2ef25e054b619068baf878bead18)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:09.966324 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:10.013191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7f2ef25e054b619068baf878bead18', 'request_id': 'req-28a75751-cc20-4c7c-92c9-211b489a4f79'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:10.019281 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7f2ef25e054b619068baf878bead18', 'request_id': 'req-28a75751-cc20-4c7c-92c9-211b489a4f79'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:10.027343 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7f2ef25e054b619068baf878bead18', 'request_id': 'req-28a75751-cc20-4c7c-92c9-211b489a4f79'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:10.038284 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] The token cache is being invalidate because project bd7f2ef25e054b619068baf878bead18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:38:10.038564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:10.039181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-28a75751-cc20-4c7c-92c9-211b489a4f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:10.040693 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1420/2854] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:38:09 2022] DELETE /identity/v3/projects/bd7f2ef25e054b619068baf878bead18 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:10.045794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb638ff2-0e1c-407c-ba85-ac071ac82790 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.046079 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb638ff2-0e1c-407c-ba85-ac071ac82790 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.046364 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb638ff2-0e1c-407c-ba85-ac071ac82790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.066095 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-bb638ff2-0e1c-407c-ba85-ac071ac82790 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:10.067309 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-bb638ff2-0e1c-407c-ba85-ac071ac82790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:10.135681 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1435/2855] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:10.137475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe8afc-122e-4ed5-aed5-8eea10adcd99 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.137768 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe8afc-122e-4ed5-aed5-8eea10adcd99 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.138181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe8afc-122e-4ed5-aed5-8eea10adcd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.158608 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-54fe8afc-122e-4ed5-aed5-8eea10adcd99 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:10.159872 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-54fe8afc-122e-4ed5-aed5-8eea10adcd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:10.169182 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1421/2856] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:10.173256 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb638ff2-0e1c-407c-ba85-ac071ac82790 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:10.183341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb638ff2-0e1c-407c-ba85-ac071ac82790 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:10.188167 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ab099-f393-46a0-bc45-3c332d5c02b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:10.188802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ab099-f393-46a0-bc45-3c332d5c02b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.189081 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ab099-f393-46a0-bc45-3c332d5c02b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.189359 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ab099-f393-46a0-bc45-3c332d5c02b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.190026 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ab099-f393-46a0-bc45-3c332d5c02b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:38:10.190330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ab099-f393-46a0-bc45-3c332d5c02b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:10.191991 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ab099-f393-46a0-bc45-3c332d5c02b3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:10.208396 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1436/2857] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:38:10.212757 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe8afc-122e-4ed5-aed5-8eea10adcd99 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:10.222426 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe8afc-122e-4ed5-aed5-8eea10adcd99 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:10.227084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34bf5c-a227-4153-9f0f-9fbea5125959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:10.227719 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34bf5c-a227-4153-9f0f-9fbea5125959 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.228019 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34bf5c-a227-4153-9f0f-9fbea5125959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.228289 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34bf5c-a227-4153-9f0f-9fbea5125959 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.228924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34bf5c-a227-4153-9f0f-9fbea5125959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:10.230803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34bf5c-a227-4153-9f0f-9fbea5125959 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:10.247865 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:10.261690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1422/2858] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:38:10 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:10.265894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ab099-f393-46a0-bc45-3c332d5c02b3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:10.276589 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ab099-f393-46a0-bc45-3c332d5c02b3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:10.331602 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abe2fa-f35c-4fdf-9d61-11d7fe36ba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:10.332230 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abe2fa-f35c-4fdf-9d61-11d7fe36ba90 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.332511 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abe2fa-f35c-4fdf-9d61-11d7fe36ba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.332828 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7abe2fa-f35c-4fdf-9d61-11d7fe36ba90 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.333945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abe2fa-f35c-4fdf-9d61-11d7fe36ba90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:10.335912 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7abe2fa-f35c-4fdf-9d61-11d7fe36ba90 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:10.344092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:10.359323 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1437/2859] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:38:10 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:10.363490 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34bf5c-a227-4153-9f0f-9fbea5125959 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:10.373925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34bf5c-a227-4153-9f0f-9fbea5125959 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:10.378805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b69b5-b742-4cea-9f17-069e5fa5e6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:10.380072 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b69b5-b742-4cea-9f17-069e5fa5e6ef admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.380323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b69b5-b742-4cea-9f17-069e5fa5e6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.380595 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b69b5-b742-4cea-9f17-069e5fa5e6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.381233 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b69b5-b742-4cea-9f17-069e5fa5e6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:10.382860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b69b5-b742-4cea-9f17-069e5fa5e6ef admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:10.393868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1423/2860] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:38:10.398552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abe2fa-f35c-4fdf-9d61-11d7fe36ba90 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:10.408485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7abe2fa-f35c-4fdf-9d61-11d7fe36ba90 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:10.413092 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6494d2-3ece-4ce3-ba4b-27b1db3f9199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:10.413682 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6494d2-3ece-4ce3-ba4b-27b1db3f9199 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.413959 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6494d2-3ece-4ce3-ba4b-27b1db3f9199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.414229 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6494d2-3ece-4ce3-ba4b-27b1db3f9199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.414863 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6494d2-3ece-4ce3-ba4b-27b1db3f9199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:10.416493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6494d2-3ece-4ce3-ba4b-27b1db3f9199 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:10.426781 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1438/2861] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:38:10.431046 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b69b5-b742-4cea-9f17-069e5fa5e6ef admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:10.441002 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b69b5-b742-4cea-9f17-069e5fa5e6ef admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:10.446055 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054f7fc-2af2-40fe-923e-e5c0209826ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:10.446698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054f7fc-2af2-40fe-923e-e5c0209826ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.446979 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054f7fc-2af2-40fe-923e-e5c0209826ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.447258 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054f7fc-2af2-40fe-923e-e5c0209826ec admin admin] PATH_INFO: `/v3/projects/a66543d6a4cc405d936e54c420220792/users/ef9b2b64cc854b4ea048e0772baca5fa/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.462264 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054f7fc-2af2-40fe-923e-e5c0209826ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66543d6a4cc405d936e54c420220792, user_id=ef9b2b64cc854b4ea048e0772baca5fa, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:10.465265 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054f7fc-2af2-40fe-923e-e5c0209826ec admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:10.480600 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1424/2862] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:38:10 2022] PUT /identity/v3/projects/a66543d6a4cc405d936e54c420220792/users/ef9b2b64cc854b4ea048e0772baca5fa/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:10.492121 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65172c-d159-4d1e-a9ab-57d10678d2a9 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.492434 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65172c-d159-4d1e-a9ab-57d10678d2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.492710 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65172c-d159-4d1e-a9ab-57d10678d2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.520301 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-3c65172c-d159-4d1e-a9ab-57d10678d2a9 None None] MFA Rules not processed for user `ef9b2b64cc854b4ea048e0772baca5fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:10.522055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3c65172c-d159-4d1e-a9ab-57d10678d2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:10.586252 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1439/2863] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:38:10.598847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054f7fc-2af2-40fe-923e-e5c0209826ec admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:10.600635 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4054f7fc-2af2-40fe-923e-e5c0209826ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:10.674709 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054f7fc-2af2-40fe-923e-e5c0209826ec admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:10.680895 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bff4a4c-4ac0-4549-9f8c-f855a622e2fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '463487449c1d4aada1b5f279c959b0e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:10.681499 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff4a4c-4ac0-4549-9f8c-f855a622e2fd service nova] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:10.681781 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff4a4c-4ac0-4549-9f8c-f855a622e2fd service nova] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:10.682057 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff4a4c-4ac0-4549-9f8c-f855a622e2fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:10.689812 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff4a4c-4ac0-4549-9f8c-f855a622e2fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:10.691386 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bff4a4c-4ac0-4549-9f8c-f855a622e2fd service nova] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:10.700745 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1425/2864] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:38:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:38:20.111165 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65172c-d159-4d1e-a9ab-57d10678d2a9 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:20.176995 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65172c-d159-4d1e-a9ab-57d10678d2a9 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:20.181528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:20.182156 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:20.182431 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:20.182699 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] PATH_INFO: `/v3/users/ef9b2b64cc854b4ea048e0772baca5fa` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:20.184353 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef9b2b64cc854b4ea048e0772baca5fa)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:20.186268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:20.233902 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef9b2b64cc854b4ea048e0772baca5fa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.241785 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef9b2b64cc854b4ea048e0772baca5fa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.242458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef9b2b64cc854b4ea048e0772baca5fa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.243109 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef9b2b64cc854b4ea048e0772baca5fa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.250244 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef9b2b64cc854b4ea048e0772baca5fa'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.256754 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:20.272394 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1440/2865] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:38:20 2022] DELETE /identity/v3/users/ef9b2b64cc854b4ea048e0772baca5fa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:38:20.279262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4ea6-e8e5-4faf-85f8-f278309391f7 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:20.279829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4ea6-e8e5-4faf-85f8-f278309391f7 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:20.279829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f4ea6-e8e5-4faf-85f8-f278309391f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:20.300743 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-212f4ea6-e8e5-4faf-85f8-f278309391f7 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:20.301953 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-212f4ea6-e8e5-4faf-85f8-f278309391f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:20.365169 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1426/2866] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:20.375386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:20.376862 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:20.436388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0fc41-74df-4c1c-b8a2-cca496a0c726 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:20.439952 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c3c6f6-e7cb-4d39-bdac-92b6ff170593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:20.440552 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3c6f6-e7cb-4d39-bdac-92b6ff170593 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:20.440832 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3c6f6-e7cb-4d39-bdac-92b6ff170593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:20.441106 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c3c6f6-e7cb-4d39-bdac-92b6ff170593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:20.450562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3c6f6-e7cb-4d39-bdac-92b6ff170593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:20.452031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c3c6f6-e7cb-4d39-bdac-92b6ff170593 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:20.461362 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1441/2867] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:38:20.732560 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f4ea6-e8e5-4faf-85f8-f278309391f7 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:20.734246 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-212f4ea6-e8e5-4faf-85f8-f278309391f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:20.750465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f4ea6-e8e5-4faf-85f8-f278309391f7 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:20.755269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:20.755946 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:20.756230 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:20.756503 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] PATH_INFO: `/v3/projects/a66543d6a4cc405d936e54c420220792` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:20.757664 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66543d6a4cc405d936e54c420220792)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:20.759576 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:20.806148 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:20.813210 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66543d6a4cc405d936e54c420220792', 'request_id': 'req-27dffc30-6740-413e-b4e1-8023fb35a413'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.820228 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66543d6a4cc405d936e54c420220792', 'request_id': 'req-27dffc30-6740-413e-b4e1-8023fb35a413'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.828680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66543d6a4cc405d936e54c420220792', 'request_id': 'req-27dffc30-6740-413e-b4e1-8023fb35a413'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.839427 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] The token cache is being invalidate because project a66543d6a4cc405d936e54c420220792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:38:20.839728 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.840349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-27dffc30-6740-413e-b4e1-8023fb35a413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:20.841803 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1427/2868] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:38:20 2022] DELETE /identity/v3/projects/a66543d6a4cc405d936e54c420220792 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:38:20.846749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d185fe-bbc6-42e7-9ff8-756727b9040b None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:20.847033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d185fe-bbc6-42e7-9ff8-756727b9040b None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:20.847308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d185fe-bbc6-42e7-9ff8-756727b9040b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:20.867519 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-f9d185fe-bbc6-42e7-9ff8-756727b9040b None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:20.868772 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-f9d185fe-bbc6-42e7-9ff8-756727b9040b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:20.930232 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1442/2869] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:20.938183 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb130a-d626-4232-847a-8d16af620d51 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:20.938472 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb130a-d626-4232-847a-8d16af620d51 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:20.938773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb130a-d626-4232-847a-8d16af620d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:20.958189 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-43eb130a-d626-4232-847a-8d16af620d51 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:20.959398 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-43eb130a-d626-4232-847a-8d16af620d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:20.967574 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1428/2870] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:20.971685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d185fe-bbc6-42e7-9ff8-756727b9040b None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:20.981820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d185fe-bbc6-42e7-9ff8-756727b9040b None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:20.986325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee942ab-328b-4826-a8ee-5d45a3fecb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:20.986922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee942ab-328b-4826-a8ee-5d45a3fecb13 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:20.987208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee942ab-328b-4826-a8ee-5d45a3fecb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:20.987482 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee942ab-328b-4826-a8ee-5d45a3fecb13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:20.988185 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee942ab-328b-4826-a8ee-5d45a3fecb13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:38:20.988486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee942ab-328b-4826-a8ee-5d45a3fecb13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:20.990676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee942ab-328b-4826-a8ee-5d45a3fecb13 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.002040 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1443/2871] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:38:21.006195 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb130a-d626-4232-847a-8d16af620d51 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.015463 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb130a-d626-4232-847a-8d16af620d51 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.021315 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7c049-7ef4-4bbc-a4ca-f7b031b3082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.021914 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7c049-7ef4-4bbc-a4ca-f7b031b3082c admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.022194 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7c049-7ef4-4bbc-a4ca-f7b031b3082c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.022465 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7c049-7ef4-4bbc-a4ca-f7b031b3082c admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.023099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7c049-7ef4-4bbc-a4ca-f7b031b3082c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.025020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7c049-7ef4-4bbc-a4ca-f7b031b3082c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.053231 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1429/2872] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:38:21 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:21.057375 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee942ab-328b-4826-a8ee-5d45a3fecb13 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.068013 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee942ab-328b-4826-a8ee-5d45a3fecb13 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.124673 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432e3b4-98ca-4d1c-8388-ac4d448bdfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.125383 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432e3b4-98ca-4d1c-8388-ac4d448bdfa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.125543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432e3b4-98ca-4d1c-8388-ac4d448bdfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.125815 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432e3b4-98ca-4d1c-8388-ac4d448bdfa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.126448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432e3b4-98ca-4d1c-8388-ac4d448bdfa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.128402 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432e3b4-98ca-4d1c-8388-ac4d448bdfa9 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.149486 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1444/2873] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:38:21 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:21.153625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7c049-7ef4-4bbc-a4ca-f7b031b3082c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.164555 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7c049-7ef4-4bbc-a4ca-f7b031b3082c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.169459 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e0ad7-0e0f-491e-886a-2f346c311428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.170054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e0ad7-0e0f-491e-886a-2f346c311428 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.170332 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e0ad7-0e0f-491e-886a-2f346c311428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.170603 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e0ad7-0e0f-491e-886a-2f346c311428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.171234 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e0ad7-0e0f-491e-886a-2f346c311428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.172853 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e0ad7-0e0f-491e-886a-2f346c311428 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.184168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1430/2874] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:38:21.188262 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432e3b4-98ca-4d1c-8388-ac4d448bdfa9 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.198782 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432e3b4-98ca-4d1c-8388-ac4d448bdfa9 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.203308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350bac8-1fbc-48f8-8f7e-9e390aad923c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.203922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350bac8-1fbc-48f8-8f7e-9e390aad923c admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.204199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350bac8-1fbc-48f8-8f7e-9e390aad923c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.204470 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350bac8-1fbc-48f8-8f7e-9e390aad923c admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.205094 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350bac8-1fbc-48f8-8f7e-9e390aad923c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.206701 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3350bac8-1fbc-48f8-8f7e-9e390aad923c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.207979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:21.219393 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1445/2875] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:38:21.222790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e0ad7-0e0f-491e-886a-2f346c311428 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.232681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e0ad7-0e0f-491e-886a-2f346c311428 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.237212 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbc07c-e6a1-4355-b04b-9b0cf0e18719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.237858 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbc07c-e6a1-4355-b04b-9b0cf0e18719 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.238140 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbc07c-e6a1-4355-b04b-9b0cf0e18719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.238417 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbc07c-e6a1-4355-b04b-9b0cf0e18719 admin admin] PATH_INFO: `/v3/projects/e0dbe7dc62694fd6930f513aaa74c428/users/abb11511c54c4b9bad6d4d3a08a0190c/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.253467 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbc07c-e6a1-4355-b04b-9b0cf0e18719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0dbe7dc62694fd6930f513aaa74c428, user_id=abb11511c54c4b9bad6d4d3a08a0190c, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.256827 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbc07c-e6a1-4355-b04b-9b0cf0e18719 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.269930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1431/2876] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:38:21 2022] PUT /identity/v3/projects/e0dbe7dc62694fd6930f513aaa74c428/users/abb11511c54c4b9bad6d4d3a08a0190c/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:38:21.280645 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac35e1d-ac76-4e4e-b538-c84fed7a4e61 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.280934 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac35e1d-ac76-4e4e-b538-c84fed7a4e61 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.281208 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac35e1d-ac76-4e4e-b538-c84fed7a4e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.309342 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-1ac35e1d-ac76-4e4e-b538-c84fed7a4e61 None None] MFA Rules not processed for user `abb11511c54c4b9bad6d4d3a08a0190c`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:21.310652 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-1ac35e1d-ac76-4e4e-b538-c84fed7a4e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:21.373929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1446/2877] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:21.378425 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbc07c-e6a1-4355-b04b-9b0cf0e18719 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.440556 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbc07c-e6a1-4355-b04b-9b0cf0e18719 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.445176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3279671-66c4-44e1-9bad-157fe4760b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.445775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3279671-66c4-44e1-9bad-157fe4760b15 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.446053 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3279671-66c4-44e1-9bad-157fe4760b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.446325 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3279671-66c4-44e1-9bad-157fe4760b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.446948 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3279671-66c4-44e1-9bad-157fe4760b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.448871 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3279671-66c4-44e1-9bad-157fe4760b15 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.475400 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1432/2878] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:38:21 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:21.479543 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac35e1d-ac76-4e4e-b538-c84fed7a4e61 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.490771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac35e1d-ac76-4e4e-b538-c84fed7a4e61 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.547936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f8051-8c31-42c7-a816-9f487268de04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.548560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f8051-8c31-42c7-a816-9f487268de04 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.548840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f8051-8c31-42c7-a816-9f487268de04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.549113 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f8051-8c31-42c7-a816-9f487268de04 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.549798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f8051-8c31-42c7-a816-9f487268de04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.551742 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f8051-8c31-42c7-a816-9f487268de04 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.575125 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1447/2879] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:38:21 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:21.579208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3279671-66c4-44e1-9bad-157fe4760b15 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.589790 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3279671-66c4-44e1-9bad-157fe4760b15 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.594695 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0e5c9-fc19-4ec4-ae57-0000907cb778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.595300 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0e5c9-fc19-4ec4-ae57-0000907cb778 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.595580 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0e5c9-fc19-4ec4-ae57-0000907cb778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.595916 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0e5c9-fc19-4ec4-ae57-0000907cb778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.596560 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0e5c9-fc19-4ec4-ae57-0000907cb778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.598285 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0e5c9-fc19-4ec4-ae57-0000907cb778 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.609480 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1433/2880] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:38:21.613877 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f8051-8c31-42c7-a816-9f487268de04 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.624586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f8051-8c31-42c7-a816-9f487268de04 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.629352 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7d0c1-a1bb-48f1-82e5-3a317ce4fd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.630008 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7d0c1-a1bb-48f1-82e5-3a317ce4fd6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.630296 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7d0c1-a1bb-48f1-82e5-3a317ce4fd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.630560 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7d0c1-a1bb-48f1-82e5-3a317ce4fd6f admin admin] PATH_INFO: `/v3/projects/76d952f42485413c9ace179e3cab2938/users/6050d76414194943bddfde17e7a6ade2/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.648137 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7d0c1-a1bb-48f1-82e5-3a317ce4fd6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d952f42485413c9ace179e3cab2938, user_id=6050d76414194943bddfde17e7a6ade2, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.650833 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7d0c1-a1bb-48f1-82e5-3a317ce4fd6f admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.665103 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1448/2881] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:38:21 2022] PUT /identity/v3/projects/76d952f42485413c9ace179e3cab2938/users/6050d76414194943bddfde17e7a6ade2/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:21.669492 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0e5c9-fc19-4ec4-ae57-0000907cb778 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.680084 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0e5c9-fc19-4ec4-ae57-0000907cb778 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.717613 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:21.739489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83b21c-3b4b-4c95-be85-233c7e3b868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.740875 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b21c-3b4b-4c95-be85-233c7e3b868d admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.741099 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b21c-3b4b-4c95-be85-233c7e3b868d admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.741476 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b21c-3b4b-4c95-be85-233c7e3b868d admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.742323 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83b21c-3b4b-4c95-be85-233c7e3b868d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.744097 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c83b21c-3b4b-4c95-be85-233c7e3b868d admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.754567 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1434/2882] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:38:21.760651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7d0c1-a1bb-48f1-82e5-3a317ce4fd6f admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.770968 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7d0c1-a1bb-48f1-82e5-3a317ce4fd6f admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.775717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c50728-2871-4431-b011-a33e092e71c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.776386 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c50728-2871-4431-b011-a33e092e71c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.776676 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c50728-2871-4431-b011-a33e092e71c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.776929 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c50728-2871-4431-b011-a33e092e71c3 admin admin] PATH_INFO: `/v3/projects/76d952f42485413c9ace179e3cab2938/users/6050d76414194943bddfde17e7a6ade2/roles/6315e2d444ce4068bdec072c623e3ec6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.778820 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c50728-2871-4431-b011-a33e092e71c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d952f42485413c9ace179e3cab2938, user_id=6050d76414194943bddfde17e7a6ade2, role_id=6315e2d444ce4068bdec072c623e3ec6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.781469 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c50728-2871-4431-b011-a33e092e71c3 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:21.792277 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1449/2883] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:38:21 2022] PUT /identity/v3/projects/76d952f42485413c9ace179e3cab2938/users/6050d76414194943bddfde17e7a6ade2/roles/6315e2d444ce4068bdec072c623e3ec6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 12:38:21.803196 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9e360-a729-4923-ab55-86f76deb0254 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.803475 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9e360-a729-4923-ab55-86f76deb0254 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.803742 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9e360-a729-4923-ab55-86f76deb0254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.830805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-77f9e360-a729-4923-ab55-86f76deb0254 None None] MFA Rules not processed for user `6050d76414194943bddfde17e7a6ade2`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:21.832042 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-77f9e360-a729-4923-ab55-86f76deb0254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:21.903054 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1435/2884] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:21.914945 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c50728-2871-4431-b011-a33e092e71c3 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:21.916868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-65c50728-2871-4431-b011-a33e092e71c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:21.958338 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:21.976238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c50728-2871-4431-b011-a33e092e71c3 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:21.980373 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fc1da-2743-49b2-9a09-6f2349336c76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:21.980982 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fc1da-2743-49b2-9a09-6f2349336c76 service cinder] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:21.981260 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fc1da-2743-49b2-9a09-6f2349336c76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:21.981531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fc1da-2743-49b2-9a09-6f2349336c76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:21.990539 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fc1da-2743-49b2-9a09-6f2349336c76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:21.992036 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fc1da-2743-49b2-9a09-6f2349336c76 service cinder] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:22.001451 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1450/2885] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:38:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:38:22.085747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9e360-a729-4923-ab55-86f76deb0254 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:22.096095 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9e360-a729-4923-ab55-86f76deb0254 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:22.099683 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567623c3-2f8a-485b-bd40-3512b3efbf0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:22.100307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567623c3-2f8a-485b-bd40-3512b3efbf0a service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:22.100584 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567623c3-2f8a-485b-bd40-3512b3efbf0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:22.100854 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567623c3-2f8a-485b-bd40-3512b3efbf0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:22.157360 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567623c3-2f8a-485b-bd40-3512b3efbf0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:22.158797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567623c3-2f8a-485b-bd40-3512b3efbf0a service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:22.168151 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1436/2886] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:38:26.916523 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fc1da-2743-49b2-9a09-6f2349336c76 service cinder] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:26.984356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fc1da-2743-49b2-9a09-6f2349336c76 service cinder] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:26.990308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:26.991617 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:26.991928 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:26.992199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] PATH_INFO: `/v3/users/abb11511c54c4b9bad6d4d3a08a0190c` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:26.993325 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb11511c54c4b9bad6d4d3a08a0190c)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:26.995254 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:27.042264 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb11511c54c4b9bad6d4d3a08a0190c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.049605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb11511c54c4b9bad6d4d3a08a0190c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.050223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb11511c54c4b9bad6d4d3a08a0190c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.050868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb11511c54c4b9bad6d4d3a08a0190c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.058582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb11511c54c4b9bad6d4d3a08a0190c'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.076458 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1451/2887] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:38:26 2022] DELETE /identity/v3/users/abb11511c54c4b9bad6d4d3a08a0190c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:27.081010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18cb65-cdd8-4e39-a57b-6020998cdbcc None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:27.081470 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18cb65-cdd8-4e39-a57b-6020998cdbcc None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:27.081572 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18cb65-cdd8-4e39-a57b-6020998cdbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:27.101379 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-da18cb65-cdd8-4e39-a57b-6020998cdbcc None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:27.102580 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-da18cb65-cdd8-4e39-a57b-6020998cdbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:27.152938 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=100 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:27.161847 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1437/2888] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:27.171582 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:27.173016 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:27.229479 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dd831-3efb-4e86-8dc3-46dfe9242c33 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:27.233155 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a8f99-95ff-4a32-87b4-d3e5cf72d506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:27.233760 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a8f99-95ff-4a32-87b4-d3e5cf72d506 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:27.234043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a8f99-95ff-4a32-87b4-d3e5cf72d506 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:27.234317 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a8f99-95ff-4a32-87b4-d3e5cf72d506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:27.243901 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a8f99-95ff-4a32-87b4-d3e5cf72d506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:27.245331 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a8f99-95ff-4a32-87b4-d3e5cf72d506 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:27.254764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1452/2889] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:38:27.532355 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18cb65-cdd8-4e39-a57b-6020998cdbcc None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:27.533777 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-da18cb65-cdd8-4e39-a57b-6020998cdbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:27.545873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18cb65-cdd8-4e39-a57b-6020998cdbcc None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:27.551820 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:27.552507 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:27.552786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:27.553059 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] PATH_INFO: `/v3/projects/e0dbe7dc62694fd6930f513aaa74c428` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:27.554191 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0dbe7dc62694fd6930f513aaa74c428)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:27.556186 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:27.617456 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0dbe7dc62694fd6930f513aaa74c428', 'request_id': 'req-03f1db6c-58aa-4523-b45c-59b0ca736c24'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.626118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0dbe7dc62694fd6930f513aaa74c428', 'request_id': 'req-03f1db6c-58aa-4523-b45c-59b0ca736c24'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.640605 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0dbe7dc62694fd6930f513aaa74c428', 'request_id': 'req-03f1db6c-58aa-4523-b45c-59b0ca736c24'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.661410 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] The token cache is being invalidate because project e0dbe7dc62694fd6930f513aaa74c428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:38:27.661756 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.662326 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.663829 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1438/2890] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:38:27 2022] DELETE /identity/v3/projects/e0dbe7dc62694fd6930f513aaa74c428 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:27.667964 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a8f99-95ff-4a32-87b4-d3e5cf72d506 service neutron] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:27.672868 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:27.678839 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a8f99-95ff-4a32-87b4-d3e5cf72d506 service neutron] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:27.737287 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:27.737909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:27.738187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:27.738456 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] PATH_INFO: `/v3/users/6050d76414194943bddfde17e7a6ade2` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:27.739572 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6050d76414194943bddfde17e7a6ade2)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:27.741591 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:27.789705 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6050d76414194943bddfde17e7a6ade2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.796341 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6050d76414194943bddfde17e7a6ade2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.797076 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6050d76414194943bddfde17e7a6ade2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.797679 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6050d76414194943bddfde17e7a6ade2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.805132 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-4c1e1959-89e0-4953-8e5e-8c3ef22918e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6050d76414194943bddfde17e7a6ade2'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:27.823454 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1453/2891] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:38:27 2022] DELETE /identity/v3/users/6050d76414194943bddfde17e7a6ade2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:28.100384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:28.101942 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:28.119925 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1db6c-58aa-4523-b45c-59b0ca736c24 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:28.180190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:28.180744 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.181024 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.181296 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] PATH_INFO: `/v3/projects/76d952f42485413c9ace179e3cab2938` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.182418 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d952f42485413c9ace179e3cab2938)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:28.184347 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:28.229924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d952f42485413c9ace179e3cab2938', 'request_id': 'req-b67ed61e-db55-4495-8084-9e428cdaf728'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:28.235944 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d952f42485413c9ace179e3cab2938', 'request_id': 'req-b67ed61e-db55-4495-8084-9e428cdaf728'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:28.244317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d952f42485413c9ace179e3cab2938', 'request_id': 'req-b67ed61e-db55-4495-8084-9e428cdaf728'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:28.255244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] The token cache is being invalidate because project 76d952f42485413c9ace179e3cab2938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:38:28.255554 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:28.256349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-b67ed61e-db55-4495-8084-9e428cdaf728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:28.257836 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1439/2892] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:38:28 2022] DELETE /identity/v3/projects/76d952f42485413c9ace179e3cab2938 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:28.263498 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712feefb-6104-4d19-aa87-e31c5b738322 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.264020 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712feefb-6104-4d19-aa87-e31c5b738322 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.264140 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712feefb-6104-4d19-aa87-e31c5b738322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.283749 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-712feefb-6104-4d19-aa87-e31c5b738322 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:28.284987 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-712feefb-6104-4d19-aa87-e31c5b738322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:28.345100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1454/2893] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:28.353444 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83725b3-908c-4f38-aacd-bb4be1f5aeab None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.353729 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83725b3-908c-4f38-aacd-bb4be1f5aeab None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.354003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83725b3-908c-4f38-aacd-bb4be1f5aeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.374514 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-d83725b3-908c-4f38-aacd-bb4be1f5aeab None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:28.375702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d83725b3-908c-4f38-aacd-bb4be1f5aeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:28.383440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1440/2894] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:28.387931 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712feefb-6104-4d19-aa87-e31c5b738322 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:28.397914 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712feefb-6104-4d19-aa87-e31c5b738322 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:28.402494 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145bea0-0c36-4b03-a020-c9ef4db0022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:28.403090 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145bea0-0c36-4b03-a020-c9ef4db0022a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.403368 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145bea0-0c36-4b03-a020-c9ef4db0022a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.403638 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5145bea0-0c36-4b03-a020-c9ef4db0022a admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.404313 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145bea0-0c36-4b03-a020-c9ef4db0022a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:38:28.404615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145bea0-0c36-4b03-a020-c9ef4db0022a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:28.406201 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5145bea0-0c36-4b03-a020-c9ef4db0022a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:28.418031 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1455/2895] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:38:28.422440 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83725b3-908c-4f38-aacd-bb4be1f5aeab None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:28.432327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83725b3-908c-4f38-aacd-bb4be1f5aeab None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:28.436889 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77107df7-9e45-4b1d-a8ff-c2323bbafb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:28.437485 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77107df7-9e45-4b1d-a8ff-c2323bbafb68 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.437764 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77107df7-9e45-4b1d-a8ff-c2323bbafb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.438034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77107df7-9e45-4b1d-a8ff-c2323bbafb68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.438652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77107df7-9e45-4b1d-a8ff-c2323bbafb68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:28.440565 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77107df7-9e45-4b1d-a8ff-c2323bbafb68 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:28.445305 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:28.468860 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1441/2896] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:38:28 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:28.473050 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145bea0-0c36-4b03-a020-c9ef4db0022a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:28.483579 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5145bea0-0c36-4b03-a020-c9ef4db0022a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:28.487981 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:28.544802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc57966-750a-4d1c-8b47-7785cb9b7526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:28.545442 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc57966-750a-4d1c-8b47-7785cb9b7526 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.545983 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc57966-750a-4d1c-8b47-7785cb9b7526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.546528 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc57966-750a-4d1c-8b47-7785cb9b7526 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.547639 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc57966-750a-4d1c-8b47-7785cb9b7526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:28.550326 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc57966-750a-4d1c-8b47-7785cb9b7526 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:28.571584 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1456/2897] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:38:28 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:28.575817 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77107df7-9e45-4b1d-a8ff-c2323bbafb68 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:28.586625 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77107df7-9e45-4b1d-a8ff-c2323bbafb68 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:28.591531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfed508-932b-487a-892f-bd74edb1317f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:28.592157 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfed508-932b-487a-892f-bd74edb1317f admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.592435 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfed508-932b-487a-892f-bd74edb1317f admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.592704 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfed508-932b-487a-892f-bd74edb1317f admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.593336 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfed508-932b-487a-892f-bd74edb1317f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:28.594920 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfed508-932b-487a-892f-bd74edb1317f admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:28.605649 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1442/2898] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:38:28.609685 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc57966-750a-4d1c-8b47-7785cb9b7526 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:28.619857 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc57966-750a-4d1c-8b47-7785cb9b7526 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:28.624507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349e833-5400-4478-baf4-9a674ab09b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:28.625100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349e833-5400-4478-baf4-9a674ab09b7a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.625384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349e833-5400-4478-baf4-9a674ab09b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.625655 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349e833-5400-4478-baf4-9a674ab09b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.626327 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349e833-5400-4478-baf4-9a674ab09b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:28.627953 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349e833-5400-4478-baf4-9a674ab09b7a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:28.639199 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1457/2899] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:38:28.643359 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfed508-932b-487a-892f-bd74edb1317f admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:28.653749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfed508-932b-487a-892f-bd74edb1317f admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:28.658643 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3f3fc-a612-4fa6-91fc-c129c7c4e338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:28.659289 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3f3fc-a612-4fa6-91fc-c129c7c4e338 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.659568 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3f3fc-a612-4fa6-91fc-c129c7c4e338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.659876 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3f3fc-a612-4fa6-91fc-c129c7c4e338 admin admin] PATH_INFO: `/v3/projects/4bc670fc95c7426397f1efdaf0a3ed23/users/afdd4f7e88754be08f67e2c683fee846/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.675142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3f3fc-a612-4fa6-91fc-c129c7c4e338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc670fc95c7426397f1efdaf0a3ed23, user_id=afdd4f7e88754be08f67e2c683fee846, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:28.677839 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3f3fc-a612-4fa6-91fc-c129c7c4e338 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:28.690518 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1443/2900] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:38:28 2022] PUT /identity/v3/projects/4bc670fc95c7426397f1efdaf0a3ed23/users/afdd4f7e88754be08f67e2c683fee846/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:28.755692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f15790-f461-409c-b41f-68364b798814 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.756062 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f15790-f461-409c-b41f-68364b798814 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.756339 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f15790-f461-409c-b41f-68364b798814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.785101 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d5f15790-f461-409c-b41f-68364b798814 None None] MFA Rules not processed for user `afdd4f7e88754be08f67e2c683fee846`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:28.786354 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d5f15790-f461-409c-b41f-68364b798814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:28.850561 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1458/2901] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:28.862014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3f3fc-a612-4fa6-91fc-c129c7c4e338 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:28.863681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-56b3f3fc-a612-4fa6-91fc-c129c7c4e338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:28.941122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3f3fc-a612-4fa6-91fc-c129c7c4e338 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:28.945168 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4f782-2c2b-401d-9509-80cf3c3807e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '56a94ccca15c462c9e7a5e88b907f420', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:28.945765 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4f782-2c2b-401d-9509-80cf3c3807e6 service glance] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:28.946045 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4f782-2c2b-401d-9509-80cf3c3807e6 service glance] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:28.946317 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4f782-2c2b-401d-9509-80cf3c3807e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:28.954910 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4f782-2c2b-401d-9509-80cf3c3807e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:28.956375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4f782-2c2b-401d-9509-80cf3c3807e6 service glance] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:28.965961 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1444/2902] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:38:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:38:38.847032 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=62,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:38.961782 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:51.875037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f15790-f461-409c-b41f-68364b798814 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:51.936709 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f15790-f461-409c-b41f-68364b798814 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:51.941228 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:51.941856 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:51.942142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:51.942414 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] PATH_INFO: `/v3/users/afdd4f7e88754be08f67e2c683fee846` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:51.943527 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] RBAC: Authorizing `identity:delete_user(user_id=afdd4f7e88754be08f67e2c683fee846)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:51.946151 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:51.990369 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afdd4f7e88754be08f67e2c683fee846'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.001757 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afdd4f7e88754be08f67e2c683fee846'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.002517 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afdd4f7e88754be08f67e2c683fee846'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.003372 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afdd4f7e88754be08f67e2c683fee846'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.012041 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afdd4f7e88754be08f67e2c683fee846'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.029460 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1459/2903] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:38:51 2022] DELETE /identity/v3/users/afdd4f7e88754be08f67e2c683fee846 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:52.034181 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482d915-d917-4e9b-b26c-07a7ead5fe69 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.034680 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482d915-d917-4e9b-b26c-07a7ead5fe69 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.034786 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482d915-d917-4e9b-b26c-07a7ead5fe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.053120 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-b482d915-d917-4e9b-b26c-07a7ead5fe69 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:52.054805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b482d915-d917-4e9b-b26c-07a7ead5fe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:52.111923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1445/2904] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:52.122484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:52.123920 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:52.197273 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12632c23-9828-4e7b-bcbd-4b3b29c8625c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:52.200992 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a6c98-b4f4-427d-92a3-c1145bfa7dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:52.201592 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a6c98-b4f4-427d-92a3-c1145bfa7dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.201874 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a6c98-b4f4-427d-92a3-c1145bfa7dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.202150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a6c98-b4f4-427d-92a3-c1145bfa7dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.211089 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a6c98-b4f4-427d-92a3-c1145bfa7dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:52.212550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a6c98-b4f4-427d-92a3-c1145bfa7dd1 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:52.222187 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1460/2905] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:38:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:38:52.484001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482d915-d917-4e9b-b26c-07a7ead5fe69 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:52.485460 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-b482d915-d917-4e9b-b26c-07a7ead5fe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:52.499017 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482d915-d917-4e9b-b26c-07a7ead5fe69 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:52.505376 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:52.506094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.506495 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.506837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] PATH_INFO: `/v3/projects/4bc670fc95c7426397f1efdaf0a3ed23` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.508954 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc670fc95c7426397f1efdaf0a3ed23)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:52.510989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:52.558118 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc670fc95c7426397f1efdaf0a3ed23', 'request_id': 'req-7e93e6c0-8952-4663-b9be-aecc59834920'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.564155 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc670fc95c7426397f1efdaf0a3ed23', 'request_id': 'req-7e93e6c0-8952-4663-b9be-aecc59834920'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.571445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc670fc95c7426397f1efdaf0a3ed23', 'request_id': 'req-7e93e6c0-8952-4663-b9be-aecc59834920'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.584529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] The token cache is being invalidate because project 4bc670fc95c7426397f1efdaf0a3ed23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:38:52.584813 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.585978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-7e93e6c0-8952-4663-b9be-aecc59834920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:38:52.587466 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1446/2906] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:38:52 2022] DELETE /identity/v3/projects/4bc670fc95c7426397f1efdaf0a3ed23 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:52.592948 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540cb42-040a-4a49-9893-1f11cad30ed7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.593241 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540cb42-040a-4a49-9893-1f11cad30ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.593524 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540cb42-040a-4a49-9893-1f11cad30ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.608669 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:52.615185 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-7540cb42-040a-4a49-9893-1f11cad30ed7 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:52.617037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-7540cb42-040a-4a49-9893-1f11cad30ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:52.677566 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1461/2907] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:52.685094 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cea3d9-3486-4083-ae25-afe0af655a61 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.685384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cea3d9-3486-4083-ae25-afe0af655a61 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.685656 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cea3d9-3486-4083-ae25-afe0af655a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.706010 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-f3cea3d9-3486-4083-ae25-afe0af655a61 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:52.707208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f3cea3d9-3486-4083-ae25-afe0af655a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:52.715816 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1447/2908] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:52.720030 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540cb42-040a-4a49-9893-1f11cad30ed7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:52.730457 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540cb42-040a-4a49-9893-1f11cad30ed7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:52.735129 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8af64-9b2e-4eaf-b054-a02c5642dab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:52.735802 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8af64-9b2e-4eaf-b054-a02c5642dab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.736087 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8af64-9b2e-4eaf-b054-a02c5642dab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.736356 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a8af64-9b2e-4eaf-b054-a02c5642dab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.736997 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8af64-9b2e-4eaf-b054-a02c5642dab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:38:52.737300 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8af64-9b2e-4eaf-b054-a02c5642dab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:52.738883 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a8af64-9b2e-4eaf-b054-a02c5642dab1 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:52.750692 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1462/2909] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:38:52.754994 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cea3d9-3486-4083-ae25-afe0af655a61 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:52.764531 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cea3d9-3486-4083-ae25-afe0af655a61 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:52.769208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d48a8-4700-4af3-8ebc-ad4e0c3fc1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:52.769804 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d48a8-4700-4af3-8ebc-ad4e0c3fc1cd admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.770081 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d48a8-4700-4af3-8ebc-ad4e0c3fc1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.770349 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d48a8-4700-4af3-8ebc-ad4e0c3fc1cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.770968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d48a8-4700-4af3-8ebc-ad4e0c3fc1cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:52.773445 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d48a8-4700-4af3-8ebc-ad4e0c3fc1cd admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:52.802278 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1448/2910] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:38:52 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:52.806798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8af64-9b2e-4eaf-b054-a02c5642dab1 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:52.816753 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a8af64-9b2e-4eaf-b054-a02c5642dab1 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:52.880764 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc72e17-7c5f-4969-9093-899285eba471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:52.881465 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc72e17-7c5f-4969-9093-899285eba471 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.881698 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc72e17-7c5f-4969-9093-899285eba471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.882390 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc72e17-7c5f-4969-9093-899285eba471 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.883039 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc72e17-7c5f-4969-9093-899285eba471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:52.885007 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc72e17-7c5f-4969-9093-899285eba471 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:52.907312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1463/2911] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:38:52 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:38:52.911691 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d48a8-4700-4af3-8ebc-ad4e0c3fc1cd admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:52.922201 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d48a8-4700-4af3-8ebc-ad4e0c3fc1cd admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:52.927949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d12fe-babf-4ea7-bcfc-edceb382e0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:52.928564 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d12fe-babf-4ea7-bcfc-edceb382e0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.928849 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d12fe-babf-4ea7-bcfc-edceb382e0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.929136 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d12fe-babf-4ea7-bcfc-edceb382e0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.929773 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d12fe-babf-4ea7-bcfc-edceb382e0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:52.931364 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d12fe-babf-4ea7-bcfc-edceb382e0a0 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:52.941297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1449/2912] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:38:52.945919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc72e17-7c5f-4969-9093-899285eba471 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:52.955829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc72e17-7c5f-4969-9093-899285eba471 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:52.961043 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c30c65-d524-4a52-bb41-06e2b04e45f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:52.961636 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c30c65-d524-4a52-bb41-06e2b04e45f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.961921 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c30c65-d524-4a52-bb41-06e2b04e45f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.962191 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c30c65-d524-4a52-bb41-06e2b04e45f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:52.962826 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c30c65-d524-4a52-bb41-06e2b04e45f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:52.964720 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c30c65-d524-4a52-bb41-06e2b04e45f6 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:52.975150 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1464/2913] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 12:38:52.979176 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d12fe-babf-4ea7-bcfc-edceb382e0a0 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:52.988915 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d12fe-babf-4ea7-bcfc-edceb382e0a0 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:52.993428 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481436df-7d5a-4ce1-ae49-3939d7b3b5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:52.994061 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481436df-7d5a-4ce1-ae49-3939d7b3b5a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:52.994339 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481436df-7d5a-4ce1-ae49-3939d7b3b5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:52.994614 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481436df-7d5a-4ce1-ae49-3939d7b3b5a2 admin admin] PATH_INFO: `/v3/projects/ca26b5e220114471bae54171a291af3a/users/cdfec5fc29fb4a8ca80187e5f6d00a15/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:53.010448 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481436df-7d5a-4ce1-ae49-3939d7b3b5a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca26b5e220114471bae54171a291af3a, user_id=cdfec5fc29fb4a8ca80187e5f6d00a15, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:53.013406 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481436df-7d5a-4ce1-ae49-3939d7b3b5a2 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:53.026552 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1450/2914] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:38:52 2022] PUT /identity/v3/projects/ca26b5e220114471bae54171a291af3a/users/cdfec5fc29fb4a8ca80187e5f6d00a15/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:38:53.037962 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16269e-898a-4cb3-8672-850e387c40d7 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:53.038246 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16269e-898a-4cb3-8672-850e387c40d7 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:53.038521 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16269e-898a-4cb3-8672-850e387c40d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:53.065642 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-fc16269e-898a-4cb3-8672-850e387c40d7 None None] MFA Rules not processed for user `cdfec5fc29fb4a8ca80187e5f6d00a15`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:38:53.066919 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-fc16269e-898a-4cb3-8672-850e387c40d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:53.131448 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1465/2915] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:38:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:38:53.142899 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481436df-7d5a-4ce1-ae49-3939d7b3b5a2 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:38:53.144762 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-481436df-7d5a-4ce1-ae49-3939d7b3b5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:38:53.167110 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:38:53.203949 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481436df-7d5a-4ce1-ae49-3939d7b3b5a2 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:38:53.207557 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583c85c-0269-4fca-bdd4-3e5022042aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:38:53.208190 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583c85c-0269-4fca-bdd4-3e5022042aec service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:38:53.208472 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583c85c-0269-4fca-bdd4-3e5022042aec service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:38:53.208747 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583c85c-0269-4fca-bdd4-3e5022042aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:38:53.218312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583c85c-0269-4fca-bdd4-3e5022042aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:38:53.219805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583c85c-0269-4fca-bdd4-3e5022042aec service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:38:53.228997 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1451/2916] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:38:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:39:02.695805 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16269e-898a-4cb3-8672-850e387c40d7 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:02.700480 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:39:02.787615 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16269e-898a-4cb3-8672-850e387c40d7 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:02.792283 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:02.792913 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:02.793195 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:02.793467 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] PATH_INFO: `/v3/users/cdfec5fc29fb4a8ca80187e5f6d00a15` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:02.794595 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfec5fc29fb4a8ca80187e5f6d00a15)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:02.796550 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:02.843581 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfec5fc29fb4a8ca80187e5f6d00a15'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:02.850794 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfec5fc29fb4a8ca80187e5f6d00a15'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:02.851507 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfec5fc29fb4a8ca80187e5f6d00a15'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:02.852138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfec5fc29fb4a8ca80187e5f6d00a15'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:02.859200 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfec5fc29fb4a8ca80187e5f6d00a15'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:02.876299 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1466/2917] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:39:02 2022] DELETE /identity/v3/users/cdfec5fc29fb4a8ca80187e5f6d00a15 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:39:02.880841 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176ad63-8c76-49b4-befd-2917c916fd7a None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:02.881327 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176ad63-8c76-49b4-befd-2917c916fd7a None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:02.881429 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176ad63-8c76-49b4-befd-2917c916fd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:02.911489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-9176ad63-8c76-49b4-befd-2917c916fd7a None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:39:02.913690 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9176ad63-8c76-49b4-befd-2917c916fd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:02.972269 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1452/2918] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:39:02.982107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:02.983544 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:03.047382 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:39:03.065055 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2327da35-12ea-4498-8a46-eeb3d0e4477c admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:03.068734 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f666f9-97dd-49fa-b85f-869e5c79c127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:03.069335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f666f9-97dd-49fa-b85f-869e5c79c127 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.069616 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f666f9-97dd-49fa-b85f-869e5c79c127 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.069893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f666f9-97dd-49fa-b85f-869e5c79c127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.079493 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f666f9-97dd-49fa-b85f-869e5c79c127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:03.080957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f666f9-97dd-49fa-b85f-869e5c79c127 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:03.091280 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1467/2919] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:39:03.355794 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176ad63-8c76-49b4-befd-2917c916fd7a None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:03.357384 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-9176ad63-8c76-49b4-befd-2917c916fd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:03.373023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176ad63-8c76-49b4-befd-2917c916fd7a None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:03.377837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:03.378494 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.378797 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.379074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] PATH_INFO: `/v3/projects/ca26b5e220114471bae54171a291af3a` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.380940 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca26b5e220114471bae54171a291af3a)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:03.382923 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:03.446403 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca26b5e220114471bae54171a291af3a', 'request_id': 'req-1bc32026-5216-49f5-a811-886113206546'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:03.456530 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca26b5e220114471bae54171a291af3a', 'request_id': 'req-1bc32026-5216-49f5-a811-886113206546'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:03.465521 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca26b5e220114471bae54171a291af3a', 'request_id': 'req-1bc32026-5216-49f5-a811-886113206546'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:03.477963 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] The token cache is being invalidate because project ca26b5e220114471bae54171a291af3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:39:03.478260 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:03.478927 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-1bc32026-5216-49f5-a811-886113206546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:39:03.480486 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1453/2920] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:39:03 2022] DELETE /identity/v3/projects/ca26b5e220114471bae54171a291af3a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:39:03.486656 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65692a-2b26-4ec0-9eac-e11e8afd3f3d None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.486965 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65692a-2b26-4ec0-9eac-e11e8afd3f3d None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.487240 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65692a-2b26-4ec0-9eac-e11e8afd3f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.508130 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-2a65692a-2b26-4ec0-9eac-e11e8afd3f3d None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:39:03.509963 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-2a65692a-2b26-4ec0-9eac-e11e8afd3f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:03.575119 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1468/2921] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:39:03.582866 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f4f97-95eb-4032-89e0-95289f4c6541 None None] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.583262 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f4f97-95eb-4032-89e0-95289f4c6541 None None] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.583638 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f4f97-95eb-4032-89e0-95289f4c6541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.604248 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.auth.core [None req-055f4f97-95eb-4032-89e0-95289f4c6541 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:39:03.605468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-055f4f97-95eb-4032-89e0-95289f4c6541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:03.613402 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1454/2922] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:39:03.617637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65692a-2b26-4ec0-9eac-e11e8afd3f3d None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:03.627771 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65692a-2b26-4ec0-9eac-e11e8afd3f3d None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:03.632238 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be698-d339-43b3-b6a4-2dced17d2e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:03.632831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be698-d339-43b3-b6a4-2dced17d2e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.633107 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be698-d339-43b3-b6a4-2dced17d2e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.633651 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2be698-d339-43b3-b6a4-2dced17d2e2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.634324 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be698-d339-43b3-b6a4-2dced17d2e2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 12:39:03.634625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be698-d339-43b3-b6a4-2dced17d2e2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:03.636243 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2be698-d339-43b3-b6a4-2dced17d2e2a admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:03.648829 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1469/2923] 10.222.0.71 () {58 vars in 1153 bytes} [Wed Sep 21 12:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 12:39:03.653208 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f4f97-95eb-4032-89e0-95289f4c6541 None None] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:03.662496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f4f97-95eb-4032-89e0-95289f4c6541 None None] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:03.667823 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6b52d-0b8a-48a4-9035-279580e60e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:03.668424 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6b52d-0b8a-48a4-9035-279580e60e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.668706 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6b52d-0b8a-48a4-9035-279580e60e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.668978 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6b52d-0b8a-48a4-9035-279580e60e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.669601 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6b52d-0b8a-48a4-9035-279580e60e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:03.671489 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6b52d-0b8a-48a4-9035-279580e60e21 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:03.699142 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1455/2924] 10.222.0.71 () {60 vars in 1153 bytes} [Wed Sep 21 12:39:03 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:39:03.703312 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be698-d339-43b3-b6a4-2dced17d2e2a admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:03.713969 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2be698-d339-43b3-b6a4-2dced17d2e2a admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:03.777033 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf34c17-5e11-49ac-9395-386b9be98c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:03.777632 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf34c17-5e11-49ac-9395-386b9be98c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.777908 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf34c17-5e11-49ac-9395-386b9be98c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.778179 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf34c17-5e11-49ac-9395-386b9be98c70 admin admin] PATH_INFO: `/v3/users` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.778798 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf34c17-5e11-49ac-9395-386b9be98c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:03.780746 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf34c17-5e11-49ac-9395-386b9be98c70 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:03.802207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1470/2925] 10.222.0.71 () {60 vars in 1144 bytes} [Wed Sep 21 12:39:03 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 12:39:03.806382 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6b52d-0b8a-48a4-9035-279580e60e21 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:03.816898 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6b52d-0b8a-48a4-9035-279580e60e21 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:03.821513 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407872e5-5b4c-4408-be7a-edee6bf8e44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:03.822109 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407872e5-5b4c-4408-be7a-edee6bf8e44c admin admin] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.822390 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407872e5-5b4c-4408-be7a-edee6bf8e44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.822662 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407872e5-5b4c-4408-be7a-edee6bf8e44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.823298 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407872e5-5b4c-4408-be7a-edee6bf8e44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:03.824924 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407872e5-5b4c-4408-be7a-edee6bf8e44c admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:03.827041 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:39:03.837461 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1456/2926] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:39:03.841586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf34c17-5e11-49ac-9395-386b9be98c70 admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:03.851623 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf34c17-5e11-49ac-9395-386b9be98c70 admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:03.856838 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7323c-a0df-4b3c-af0f-9e205133b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:03.857426 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7323c-a0df-4b3c-af0f-9e205133b562 admin admin] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.857707 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7323c-a0df-4b3c-af0f-9e205133b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.857977 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7323c-a0df-4b3c-af0f-9e205133b562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.858608 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7323c-a0df-4b3c-af0f-9e205133b562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:03.860242 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7323c-a0df-4b3c-af0f-9e205133b562 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:03.870610 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1471/2927] 10.222.0.71 () {58 vars in 1122 bytes} [Wed Sep 21 12:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 12:39:03.875023 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407872e5-5b4c-4408-be7a-edee6bf8e44c admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:03.885003 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407872e5-5b4c-4408-be7a-edee6bf8e44c admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:03.889741 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae21ff8-817c-4f6b-9812-ab6e89115575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:03.890391 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae21ff8-817c-4f6b-9812-ab6e89115575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.890668 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae21ff8-817c-4f6b-9812-ab6e89115575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.890937 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae21ff8-817c-4f6b-9812-ab6e89115575 admin admin] PATH_INFO: `/v3/projects/c274a3c4e6954affbfe3d07e849d64b7/users/d757e6ac07be4c79868adee5777dc461/roles/3367be0edc604df081b5ac7a9e2a7b92` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.906020 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae21ff8-817c-4f6b-9812-ab6e89115575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c274a3c4e6954affbfe3d07e849d64b7, user_id=d757e6ac07be4c79868adee5777dc461, role_id=3367be0edc604df081b5ac7a9e2a7b92)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:03.908698 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae21ff8-817c-4f6b-9812-ab6e89115575 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:03.922056 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1457/2928] 10.222.0.71 () {60 vars in 1483 bytes} [Wed Sep 21 12:39:03 2022] PUT /identity/v3/projects/c274a3c4e6954affbfe3d07e849d64b7/users/d757e6ac07be4c79868adee5777dc461/roles/3367be0edc604df081b5ac7a9e2a7b92 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:39:03.927059 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9464ea2-e2d7-448b-9c99-c2e292b5e267 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:03.927351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9464ea2-e2d7-448b-9c99-c2e292b5e267 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:03.927654 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9464ea2-e2d7-448b-9c99-c2e292b5e267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:03.947384 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-d9464ea2-e2d7-448b-9c99-c2e292b5e267 None None] MFA Rules not processed for user `661f5d19be0f436ca14e6e9b440ca709`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:39:03.948637 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-d9464ea2-e2d7-448b-9c99-c2e292b5e267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:03.990531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:39:04.009537 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1472/2929] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:39:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:39:04.019139 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae21ff8-817c-4f6b-9812-ab6e89115575 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:04.020877 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-8ae21ff8-817c-4f6b-9812-ab6e89115575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:04.079873 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae21ff8-817c-4f6b-9812-ab6e89115575 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:04.083375 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c023d-e448-4dec-b8f1-91872e7bd384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:04.084014 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c023d-e448-4dec-b8f1-91872e7bd384 service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:04.084297 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c023d-e448-4dec-b8f1-91872e7bd384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:04.084571 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c023d-e448-4dec-b8f1-91872e7bd384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:04.094493 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c023d-e448-4dec-b8f1-91872e7bd384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:04.095945 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c023d-e448-4dec-b8f1-91872e7bd384 service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:04.105501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1458/2930] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:39:04.736538 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc206-9c57-4b46-8829-6582619985f4 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:04.737100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc206-9c57-4b46-8829-6582619985f4 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:04.737100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc206-9c57-4b46-8829-6582619985f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:04.774330 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-c5dcc206-9c57-4b46-8829-6582619985f4 None None] MFA Rules not processed for user `d757e6ac07be4c79868adee5777dc461`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:39:04.775567 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-c5dcc206-9c57-4b46-8829-6582619985f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:04.842871 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1473/2931] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:39:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:39:04.854651 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c023d-e448-4dec-b8f1-91872e7bd384 service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:39:04.856162 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-517c023d-e448-4dec-b8f1-91872e7bd384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:04.916491 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c023d-e448-4dec-b8f1-91872e7bd384 service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:39:04.920159 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f61ff-df94-435f-a3b3-d6f6b4376398 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c84b440821446e3a7ce97b2c68f815e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:39:04.920760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f61ff-df94-435f-a3b3-d6f6b4376398 service cinder] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:04.921035 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f61ff-df94-435f-a3b3-d6f6b4376398 service cinder] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:04.921306 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f61ff-df94-435f-a3b3-d6f6b4376398 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:04.929855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f61ff-df94-435f-a3b3-d6f6b4376398 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:39:04.931930 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f61ff-df94-435f-a3b3-d6f6b4376398 service cinder] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:39:04.940648 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1459/2932] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:39:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:39:06.304586 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b919d87-5cf7-41f8-b24c-c233e6cfb654 None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:39:06.305142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b919d87-5cf7-41f8-b24c-c233e6cfb654 None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:39:06.305142 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b919d87-5cf7-41f8-b24c-c233e6cfb654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:39:06.324770 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-4b919d87-5cf7-41f8-b24c-c233e6cfb654 None None] MFA Rules not processed for user `d757e6ac07be4c79868adee5777dc461`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:39:06.325960 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b919d87-5cf7-41f8-b24c-c233e6cfb654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:39:06.332801 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1474/2933] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:39:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 12:39:14.936760 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:39:16.318747 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=41 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:40:06.645071 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f61ff-df94-435f-a3b3-d6f6b4376398 service cinder] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:40:06.646590 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-d61f61ff-df94-435f-a3b3-d6f6b4376398 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:06.738346 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f61ff-df94-435f-a3b3-d6f6b4376398 service cinder] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:40:06.743775 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:40:06.744377 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:06.744652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:06.744918 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:06.746652 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:06.758837 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:40:06.760434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:40:06.770548 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1460/2934] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:40:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:40:16.766180 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=41 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:40:21.918593 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b919d87-5cf7-41f8-b24c-c233e6cfb654 None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:40:21.920219 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-4b919d87-5cf7-41f8-b24c-c233e6cfb654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:21.932985 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b919d87-5cf7-41f8-b24c-c233e6cfb654 None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:40:21.936503 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac44bb-d08b-4895-a006-f53471f73e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:40:21.937100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac44bb-d08b-4895-a006-f53471f73e27 service neutron] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:21.937377 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac44bb-d08b-4895-a006-f53471f73e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:21.937649 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac44bb-d08b-4895-a006-f53471f73e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:21.939308 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-e0ac44bb-d08b-4895-a006-f53471f73e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:21.973138 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac44bb-d08b-4895-a006-f53471f73e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:40:21.974562 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac44bb-d08b-4895-a006-f53471f73e27 service neutron] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:40:21.993495 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1475/2935] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:40:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:40:24.875288 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:40:24.877122 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:24.891855 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45eab6-2f3a-4392-a2e5-fbf68beb9ffd service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:40:24.897705 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3760f3-d87f-497e-9363-124c016e355d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:40:24.898934 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3760f3-d87f-497e-9363-124c016e355d service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:24.899321 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3760f3-d87f-497e-9363-124c016e355d service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:24.899707 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3760f3-d87f-497e-9363-124c016e355d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:24.901570 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-fb3760f3-d87f-497e-9363-124c016e355d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:24.915931 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3760f3-d87f-497e-9363-124c016e355d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:40:24.917702 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3760f3-d87f-497e-9363-124c016e355d service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:40:24.929681 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1461/2936] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:40:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:40:26.771463 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79979e0e-a5a2-4174-a453-cf161dbca03a None None] REQUEST_METHOD: `POST` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:26.771955 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79979e0e-a5a2-4174-a453-cf161dbca03a None None] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:26.772085 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79979e0e-a5a2-4174-a453-cf161dbca03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:26.791611 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.auth.core [None req-79979e0e-a5a2-4174-a453-cf161dbca03a None None] MFA Rules not processed for user `d757e6ac07be4c79868adee5777dc461`. Rule list: `[]` (Enabled: `True`). {{(pid=81406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 12:40:26.792894 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-79979e0e-a5a2-4174-a453-cf161dbca03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:26.800223 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1476/2937] 10.222.0.71 () {58 vars in 958 bytes} [Wed Sep 21 12:40:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 12:40:27.174993 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3760f3-d87f-497e-9363-124c016e355d service placement] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:40:27.185342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3760f3-d87f-497e-9363-124c016e355d service placement] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:40:27.189006 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd66d5d9989449729cec7e60cdaf0949', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:40:27.189678 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:27.190026 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:27.190363 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:27.192452 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:27.204529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:40:27.206666 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:40:27.221589 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1462/2938] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:40:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:40:28.404939 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79979e0e-a5a2-4174-a453-cf161dbca03a None None] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:40:28.419504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79979e0e-a5a2-4174-a453-cf161dbca03a None None] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:40:28.426259 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64dab9-8b59-437e-9503-2d6da5dc454e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:40:28.427170 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64dab9-8b59-437e-9503-2d6da5dc454e service placement] REQUEST_METHOD: `GET` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:28.427678 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64dab9-8b59-437e-9503-2d6da5dc454e service placement] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:28.428037 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64dab9-8b59-437e-9503-2d6da5dc454e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:28.430207 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-8a64dab9-8b59-437e-9503-2d6da5dc454e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:28.443214 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64dab9-8b59-437e-9503-2d6da5dc454e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:40:28.445093 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64dab9-8b59-437e-9503-2d6da5dc454e service placement] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:40:28.459233 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1477/2939] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 12:40:28.473943 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:40:28.475515 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:28.496154 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469743c9-9f6a-4cff-83dd-427138058f1f service neutron] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:40:28.501468 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:40:28.502197 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:28.502498 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:28.502733 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] PATH_INFO: `/v3/users/f84520fd4b434d99ad0dab1a1dd3b033` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:28.504434 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84520fd4b434d99ad0dab1a1dd3b033)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:40:28.506628 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:40:28.555749 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84520fd4b434d99ad0dab1a1dd3b033'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.556616 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84520fd4b434d99ad0dab1a1dd3b033'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.564620 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84520fd4b434d99ad0dab1a1dd3b033'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.565221 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84520fd4b434d99ad0dab1a1dd3b033'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.572583 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84520fd4b434d99ad0dab1a1dd3b033'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.589968 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1463/2940] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:40:28 2022] DELETE /identity/v3/users/f84520fd4b434d99ad0dab1a1dd3b033 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:40:28.760810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64dab9-8b59-437e-9503-2d6da5dc454e service placement] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:40:28.773003 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64dab9-8b59-437e-9503-2d6da5dc454e service placement] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:40:28.838047 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:40:28.838768 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:28.839080 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:28.839358 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] PATH_INFO: `/v3/projects/04bda0b59d3b4bfabf6f1b84fec0d6a6` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:28.840943 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=04bda0b59d3b4bfabf6f1b84fec0d6a6)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:40:28.844100 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:40:28.919549 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04bda0b59d3b4bfabf6f1b84fec0d6a6', 'request_id': 'req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.929175 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04bda0b59d3b4bfabf6f1b84fec0d6a6', 'request_id': 'req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.939898 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04bda0b59d3b4bfabf6f1b84fec0d6a6', 'request_id': 'req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.958361 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] The token cache is being invalidate because project 04bda0b59d3b4bfabf6f1b84fec0d6a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:40:28.959425 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.960630 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:28.964060 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1478/2941] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:40:28 2022] DELETE /identity/v3/projects/04bda0b59d3b4bfabf6f1b84fec0d6a6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:40:28.968307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:40:28.970348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:29.007798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa9cf6-6554-498b-89e9-11793fc1e1fa admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:40:29.094342 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:40:29.095001 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:29.095361 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:29.095687 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] PATH_INFO: `/v3/users/ddbcf7b849554d6b9b2dd4f497e4b6fa` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:29.096984 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddbcf7b849554d6b9b2dd4f497e4b6fa)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:40:29.098821 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:40:29.158385 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddbcf7b849554d6b9b2dd4f497e4b6fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.159074 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddbcf7b849554d6b9b2dd4f497e4b6fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.161989 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:40:29.180882 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddbcf7b849554d6b9b2dd4f497e4b6fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.181573 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddbcf7b849554d6b9b2dd4f497e4b6fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.189146 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddbcf7b849554d6b9b2dd4f497e4b6fa'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.208340 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1464/2942] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:40:28 2022] DELETE /identity/v3/users/ddbcf7b849554d6b9b2dd4f497e4b6fa => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:40:29.384609 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:40:29.386335 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:40:29.402484 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cf21c-0f39-4278-a945-f9f5ea2010bd admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:40:29.432268 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:40:29.467557 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:40:29.468261 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:40:29.468542 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:40:29.468816 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] PATH_INFO: `/v3/projects/aee7cc95fa89440a9d6e270a694f04b0` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:40:29.469957 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee7cc95fa89440a9d6e270a694f04b0)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:40:29.471909 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:40:29.528504 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee7cc95fa89440a9d6e270a694f04b0', 'request_id': 'req-30d1e36d-18e4-4c01-b926-679ed55b6c9e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.537644 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee7cc95fa89440a9d6e270a694f04b0', 'request_id': 'req-30d1e36d-18e4-4c01-b926-679ed55b6c9e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.546936 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee7cc95fa89440a9d6e270a694f04b0', 'request_id': 'req-30d1e36d-18e4-4c01-b926-679ed55b6c9e'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.558688 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] The token cache is being invalidate because project aee7cc95fa89440a9d6e270a694f04b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:40:29.558979 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.559605 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:40:29.561349 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1479/2943] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:40:29 2022] DELETE /identity/v3/projects/aee7cc95fa89440a9d6e270a694f04b0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:40:39.204661 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:40:39.555477 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:41:30.270348 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:41:30.271911 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:41:30.286478 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbc7a7-8d4f-4c6a-adea-3474f74016d9 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:41:30.342034 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:41:30.342655 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:41:30.342928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:41:30.343192 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] PATH_INFO: `/v3/users/d757e6ac07be4c79868adee5777dc461` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:41:30.344312 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d757e6ac07be4c79868adee5777dc461)` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:41:30.346200 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:41:30.390868 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd757e6ac07be4c79868adee5777dc461'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.391529 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd757e6ac07be4c79868adee5777dc461'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.399244 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd757e6ac07be4c79868adee5777dc461'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.399886 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd757e6ac07be4c79868adee5777dc461'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.407509 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.notifications [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd757e6ac07be4c79868adee5777dc461'} {{(pid=81407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.424271 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1465/2944] 10.222.0.71 () {58 vars in 1224 bytes} [Wed Sep 21 12:41:30 2022] DELETE /identity/v3/users/d757e6ac07be4c79868adee5777dc461 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:41:30.601004 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] Authenticating user token {{(pid=81406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:41:30.602625 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.fernet_utils [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:41:30.616351 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1e36d-18e4-4c01-b926-679ed55b6c9e admin admin] Validating token access rules against request {{(pid=81406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:41:30.678010 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '661f5d19be0f436ca14e6e9b440ca709', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5d4deb4c3b0e444f92c029c040b5603c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:41:30.678717 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:41:30.678922 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:41:30.679188 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] PATH_INFO: `/v3/projects/c274a3c4e6954affbfe3d07e849d64b7` {{(pid=81406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:41:30.680893 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c274a3c4e6954affbfe3d07e849d64b7)` {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:41:30.682831 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] RBAC: Authorization granted {{(pid=81406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:41:30.731429 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c274a3c4e6954affbfe3d07e849d64b7', 'request_id': 'req-a7013c34-d322-48b8-b208-3a8bbd2e1a87'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.737531 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c274a3c4e6954affbfe3d07e849d64b7', 'request_id': 'req-a7013c34-d322-48b8-b208-3a8bbd2e1a87'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.748716 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c274a3c4e6954affbfe3d07e849d64b7', 'request_id': 'req-a7013c34-d322-48b8-b208-3a8bbd2e1a87'} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.761810 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] The token cache is being invalidate because project c274a3c4e6954affbfe3d07e849d64b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 12:41:30.762840 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.763485 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG keystone.notifications [None req-a7013c34-d322-48b8-b208-3a8bbd2e1a87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 12:41:30.765258 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: [pid: 81406|app: 0|req: 1480/2945] 10.222.0.71 () {58 vars in 1233 bytes} [Wed Sep 21 12:41:30 2022] DELETE /identity/v3/projects/c274a3c4e6954affbfe3d07e849d64b7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 12:41:40.420807 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:41:40.756388 dv-cinder-858678-1-iscsi devstack@keystone.service[81406]: DEBUG dbcounter [-] [81406] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 12:45:08.804087 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] Authenticating user token {{(pid=81407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 12:45:08.805496 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:45:08.874085 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303c8ad-020e-4797-860a-215f1664c7b4 admin admin] Validating token access rules against request {{(pid=81407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 12:45:08.933928 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7da293-9cde-4381-9013-59b6651a45af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eac29fd43f1f4bbd987950b90385200b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5918af5b051047a78ca6d5f0187d5156', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 12:45:08.934523 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7da293-9cde-4381-9013-59b6651a45af service placement] REQUEST_METHOD: `GET` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 12:45:08.934798 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7da293-9cde-4381-9013-59b6651a45af service placement] SCRIPT_NAME: `/identity` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 12:45:08.935073 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7da293-9cde-4381-9013-59b6651a45af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 12:45:08.936805 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.fernet_utils [None req-4e7da293-9cde-4381-9013-59b6651a45af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 12:45:08.947800 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7da293-9cde-4381-9013-59b6651a45af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 12:45:08.949307 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7da293-9cde-4381-9013-59b6651a45af service placement] RBAC: Authorization granted {{(pid=81407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 12:45:08.959501 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: [pid: 81407|app: 0|req: 1466/2946] 10.222.0.71 () {60 vars in 1365 bytes} [Wed Sep 21 12:45:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 12:45:18.954806 dv-cinder-858678-1-iscsi devstack@keystone.service[81407]: DEBUG dbcounter [-] [81407] Writing DB stats keystone:SELECT=58 {{(pid=81407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}